Inlining Java Native Calls at Runtime
Inlining Java Native Calls at Runtime
ABSTRACT
We introduce a strategy for inlining native functions into
JavaTM applications using a JIT compiler. We perform fur-
ther optimizations to transform inlined callbacks into seman-
tically equivalent lightweight operations. We show that this
strategy can substantially reduce the overhead of perform-
ing JNI calls, while preserving the key safety and portabil-
ity properties of the JNI. Our work leverages the ability to
store statically-generated IL alongside native binaries, to fa-
cilitate native inlining at Java callsites at JIT compilation
time. Preliminary results with our prototype implementa-
tion show speedups of up to 93X when inlining and callback
Figure 1: The Java Native Interface
transformation are combined.
121
mechanisms. For example, at each callout to native code, Ogawa et al. [28], and the MicrosoftTM Marmot JVM Class
extra data (beyond the actual arguments themselves) must Libraries [12].
be marshaled into the call to provide the native code with Strong arguments in favor of implementing numerical
access to the JVM. routines in Java native functions are made by Bik and
Our work addresses these overheads directly by providing Gannon [6]; despite improvements in pure Java numerical
a Just-in-Time (JIT) compiler optimization targeting native libraries, interfaces to widely-used but platform-dependent
function calls. Specifically, we extend the JIT compiler’s optimized native linear algebra packages are still being de-
function inlining optimization to handle callouts to native veloped [17]. The graphical components of Java-based
functions, and provide a transformation mechanism to deal user interface libraries, including the Standard Widgets Tool-
with inlined JNI callbacks. We use inlining to reduce the kit [27] and the Abstract Window Toolkit [31], as well as
number of callouts, and then take advantage of the JVM other Java-based multimedia APIs [24] rely on the JNI to
context to transform JNI callbacks from overhead-laden op- make use of underlying architecture functionality that isn’t
erations to semantically equivalent, but dramatically less provided in Java. Native codes are also used to recover other
expensive operations. In addition, we believe that making functionality unavailable in Java, including low-level hard-
native function callsites more transparent will expose them ware performance measuring tools [29], and accurate timers
to further optimization opportunities in our JIT compiler. and system resource monitors [5].
Strategies for function inlining, whether for traditional The JNI provides much-needed access to low-level native
statically-compiled code, for dynamic JIT compilation, or code, but the overhead of this interface is significant.
for polymorphic callsites, are well-researched areas of com-
piler optimization, and are not the focus of this paper. The 2.2 JNI Performance Issues
main contribution of this work is to extend inlining to na- The JNI’s strength lies in decoupling native codes from a
tive functions during JIT compilation. For our purposes, we specific JVM implementation by providing relatively opaque
simply require support for general function inlining in the access to JVM internals, data, and services. The cost of this
JIT compiler framework. property is lost efficiency, namely large runtime overheads
The rest of the paper is organized as follows. In Section 2, during callouts to native functions, and even larger ones
we motivate our work by showcasing the ubiquity of native during callbacks to access Java code and data.
functions and the JNI in Java programs. We also provide
more details on the overheads currently affecting the us- 2.2.1 Cost of Callouts to Native Code
age of both, and introduce our approach. Section 3 then Generally speaking, a native library containing the func-
describes the JVM and JIT compiler framework that our tion being invoked must be loaded on or before the first in-
work builds on, and Section 4 describes the design of our voke Bytecode is expected, the containing class and function
proposed solution. The current status of our prototype im- must be resolved (resolution may require multiple passes
plementation and initial results are presented in Section 5. over the exported functions in a native library), and related
We compare our approach to improving JNI performance to JVM data structures must be updated. These are one-time
prior strategies, and discuss other related work on language costs that can be amortized if a particular native function
interoperability in Section 6. We then conclude in Sections 7 is invoked repeatedly.
and 8 by summarizing various issues that need addressing During each individual native function invocation, the
as our work progresses. JVM must also perform work to set up the native stack and
possibly registers in order to pass arguments, and must per-
2. BACKGROUND AND MOTIVATION form null-checking on reference arguments before possibly
The Java programming language provides desirable high- adding a layer of indirection to them. Upon returning from
level features, including platform independence, type safety, native code, the return value must be pushed onto the Java
object orientation, and automatic memory management via stack and the native stack must be restored. In addition, ex-
garbage collection, which have led to its widespread adop- ception status must be checked and local reference cleanup
tion in many settings. The JNI, a required feature of all performed on return. JVMs with JIT compilers may reduce
specification-conforming JVMs, greatly enhances the appli- these overheads by generating specialized code segments to
cability of Java by allowing legacy, high-performance, or perform some of the required work at native call sites, as is
architecture-dependent native codes to be integrated with done in the IntelTM Open Runtime Platform [10] and in our
Java applications. We begin this section by highlighting JIT compiler (see Section 3.1).
some of the uses of the JNI, motivating the importance of Sunderam and Kurzinyec [33] studied the performance of
this interface for a large number of applications. We then different types of native calls using different JVM imple-
discuss the performance issues associated with use of the mentations. Slowdowns when using native functions ranged
JNI, and introduce our strategy for addressing these issues. from a factor of 1.05 to a factor of 16 in the worst case.
Only one case achieved a speedup during a native call. Sim-
2.1 JNI: The Pervasive API ilar results are produced in overhead-measuring experiments
JNI calls occur in many different classes of applications. performed by Murray et al. [25]. These high overheads are
They have been used in I/O implementations, including cause for concern, especially since one of the motivations for
improving performance of object serialization for distributed using the JNI is to allow access to high-performance native
computing [9], providing bindings to low-level parallel com- code.
puting libraries [4, 14], as well as for high-speed network
interfaces [35]. The JNI is also used to implement various 2.2.2 Cost of JNI Callbacks
JVM frameworks, such the Java 1.5 Class Libraries as Although callouts are reasonably expensive, the more sig-
well as the reflective, Java-based OpenJIT compiler from nificant source of interoperability overhead occurs during the
122
invocation of JNI callbacks. Because of the JNI’s platform- perform the extra work required at JNI invocation points,
independent design, JNI functions are only callable through however, we aim to eliminate much of this extra work en-
a reference to the JNI Environment variable (the JNIEnv tirely. We consider other strategies for dealing with JNI
pointer). This JNI Environment is a table of function point- overhead in more detail in Section 6.
ers, each referring to the implementation of a specific call- Our approach is to extend the JIT compiler’s function in-
back function. A callback thus pays a performance penalty lining optimization to handle native function calls. Once na-
because two levels of indirection1 are used - one to obtain the tive code has been inlined at its callsite in a Java program, it
appropriate function pointer through the JNIEnv pointer, is no longer necessary to set up and tear down a native stack,
and one to invoke the function using that pointer. Other, or perform other expensive operations to pass arguments.
more specific callback overheads depend on the JNI function More importantly, the callbacks, designed to gain access to
being called: internal JVM state, can now be transformed into compile-
time constants or lightweight Bytecodes that preserve the
• String and array parameters - To make use of semantics of the original source program. Our prototype im-
string and array data originating from a Java appli- plementation shows significant performance increases from
cation, native code must first acquire native-side ac- inlining native code and transforming JNI callbacks for sim-
cess to them. Unfortunately, this requires expensive ple microbenchmarks. Although the implementation is not
runtime copy operations whose purpose is to leave the complete, we expect these benefits to translate into perfor-
JVM’s copy of the data untouched. The JNI, how- mance improvements in real applications that make exten-
ever, also provides callbacks that claim to increase sive use of the JNI. In addition, we believe that native func-
the chances of receiving direct references to underly- tion inlining will expose native code to other optimization
ing JVM data, but this is left to the JVM’s discretion opportunities, and reduce the need for the JIT compiler to
and also places certain restrictions on the program- make conservative assumptions when optimizing Java code.
mer’s freedom. Because JVMs may implement these Our strategy for native function inlining has been proto-
callbacks in any way they choose, there is no guarantee typed and evaluated in the context of a high-performance
that better performance will actually result from their production JVM and JIT compiler from IBM. Before de-
use. Sunderam and Kurzinyec [33] demonstrate that scribing the details of our strategy, we thus first describe
the achieved performance varies widely across different this framework.
JVM implementations.
• Fields and methods - Using Java data types, modi- 3. JIT COMPILER FRAMEWORK
fying object data, calling methods and accessing JVM In this section, we introduce the VM and JIT compiler
services from native code are also performed via call- technologies from IBM that form the basis of our imple-
backs. For example, modifying a field of an object or mentation, and present the inlining strategy employed by
calling a static method first requires retrieving a han- the JIT compiler.
dle to it. This retrieval is commonly implemented as The IBM R TR JIT compiler is a high-quality, high-
a traversal on the JVM’s reflective data structures, in performance optimizing compiler, conceived and developed
addition to expensive string-based signature compari- at the IBM Toronto Software Lab. Designed with a high
son operations at runtime [8]. Results in [33] highlight level of configurability in mind, it supports multiple JVMs
these overheads: field accesses in Java are orders of and class library implementations, targets many architec-
magnitude quicker than those via the JNI. tures, can achieve various memory footprint goals and has
• Other JNI functions - The JNI also provides call- a wide range of optimizations and optimization strategies.
backs that instantiate objects, manage references, han- The control flow for the TR JIT consists of phases for in-
dle exceptions, support synchronization and reflection termediate language (IL) generation, optimization and code
as well as those that provide the ability to embed JVMs generation as depicted in Figure 2. When compiling a method,
inside native code. These functions share costs simi- the IL Generator walks the method’s Bytecodes, and gen-
lar to those of field and method access callbacks, but erates tree-based JIT compiler IL (referred to as TR-IL) that
have their own unique set of additional overheads. We also encodes the control flow graph. The Optimization
exclude analyzing them in this paper. phase is a pipeline through which the TR-IL flows and may
be modified and reordered by architecture-independent/de-
We also note that callbacks may block if a JVM is in the pendent, speculative and profile-based adaptive optimiza-
midst of performing a blocking task (such as garbage collec- tions. The Code Generation phase lowers the TR-IL to
tion) when the callback occurs. a specific machine instruction set, performs register alloca-
tion and schedules the instructions before emitting a final
2.3 Addressing JNI Overheads binary encoding. Auxiliary data structures which include
Given the extensive use of the JNI in existing applications, stack maps and exception tables are also generated at this
we believe the performance penalties must be addressed di- point.
rectly, rather than by introducing changes to the interface, The TR JIT compiler is currently used by the IBM J9
or introducing a new interoperability mechanism for Java. Java virtual machine2 . J9 is a clean-room JVM implemen-
Further, since high-performance JVMs include JIT compil- tation targeting numerous different processors and operating
ers, it is appropriate to leverage JIT optimizations to reduce systems, supporting ahead-of-time compilation, and method
the overheads incurred as a result of using the JNI for in- hot-swapping as well as a host of other features.
teroperability. Instead of simply generating efficient code to 2
performance results for a TR JIT enabled J9 virtual ma-
1
Only one level of indirection is required for C++ natives. chine can be found on [Link]
123
Figure 2: Architecture of the TR JIT compiler
124
addresses of variables (automatics or parameters) that may
be passed by an inlined native to one of its synthesized calls.
In these cases, TR is careful to ensure that correct linkage
is used and the inlined native code is able to read and write
to the same portion of shared memory as non-inlined func-
tions. Because this address resolution is performed at JIT
compile-time, and the original native function is now inlined
rather than called explicitly, additional care must be taken
to ensure that the dynamic loading of new libraries is han-
dled correctly.
Finally, the inlined native methods clearly execute in a
Java context. Therefore, the code must be conditioned to
interact with all appropriate VM requirements. In particu-
lar, instructions to perform handshaking with VM compo-
nents such as garbage collection are inserted before and after
inlined native code.
Figure 4: Inlining native codes
The inliner recursively inlines functions called by a Java-
callable native method until it either encounters a call to an
opaque function or internal code growth limits are reached.
runtime, when the inliner decides to inline a particular na- Having reached a termination point, the inliner examines the
tive function. In principle, the conversion could also be resulting IL for synthesis requirements, and after satisfying
done offline, storing a representation of the JIT compiler’s them, continues with normal JIT compiler processing.
IL along with the native executable. TR-IL, however, is an Note that inlining a single original native function call
in-memory IL and is not suitable for efficient serialization to may require the synthesis of multiple calls to opaque natives,
disk. In contrast, W-Code (like Bytecode) is a suitable disk as depicted in Figure 5. Inlining, however, creates the op-
format by design, and the conversion to TR-IL is a single- portunity to remove the much-higher overhead of callbacks,
pass, lightweight operation. Storing the more compact rep- and reduces the need for conservative assumptions about
resentation and converting at runtime is in keeping with the the behavior of native code in the JIT optimizer. We thus
slim binaries strategy proposed by Franz and Kistler [13]. expect that it will often be profitable to synthesize multiple
callouts to opaque natives, provided callbacks can be trans-
4.2 Inlining and Synthesizing Native Calls formed into cheaper operations as we discuss in the following
The native inlining proceeds as follows: TR uses the con- section.
version engine described in Section 4.1 to load and generate
TR-IL for a Java-callable native function, and then maps 4.3 Transforming JNI Callbacks
parameters to arguments, generates temporaries as needed, The native inlining process is augmented by JNI callback
merges caller and callee IL and control flow graphs, and transformations; inlined native code executes in the JVM
materializes the JNIEnv pointer for use by inlined JNI call- context, thus there is no need for the JNIEnv pointer and
backs. This inlining process is recursive and considers non- the JNI function pointer table to obtain access to inter-
Java callable native functions as potential inlineable candi- nal JVM services and data. Once the native inlining tech-
dates as well. Figure 4 depicts this process. nique has converted W-Codes to TR-IL, it pattern-matches
the generated IL, looking for JNI callbacks. This pattern-
4.2.1 Synthesized Native Functions matching step is necessary because it is impossible to dis-
The IL for a native method cannot, in non-trivial cases, tinguish a function call that performs a JNI callback from
proceed directly through the rest of JIT compiler processing. any other function call based solely on the IL. Whenever
This is because it may contain calls to non Java-callable possible, these callbacks are transformed into compile-time
natives for which the JIT compiler cannot render an IL. Such constants, and new semantically equivalent TR-IL that rep-
“opaque” calls occur in two situations: (1) calls through resents faster, more direct access to JVM services and data.
function pointers, and (2) calls to functions in binaries where This process proceeds by using a mechanism, depicted in
W-Code is unavailable. Opaque native calls are replaced Figure 6, to iterate over each TR-IL instruction and per-
with calls to synthesized methods similar to those produced form the required transformation.
by Direct2JNI, whose purpose is to call the native function
after having set up the proper linkages and context to make 4.3.1 Identifying Inlined JNI Callbacks
the call. The effect is to bridge the Java application to the Since our technique is based on transforming TR-IL and
previously-buried native function. This situation is depicted understanding the semantics of JNI callbacks, we require a
in Figure 5. In some cases (e.g., when the opaque function is preliminary step that renders each callback defined by the
a well-known library function) the call to the function can be JNI API in terms of TR-IL. This step allows the compiler to
effected without going through an entire intermediate native understand the expected “shape” of each JNI callback. The
method. “shape” encodes how each callback uses the JNIEnv pointer
Another concern is data shared between an inlined native and any other arguments. This representative shape is what
function and any other native function (i.e., a synthesized uniquely determines a callback. TR uses the constructed
native function, another function in the same library, or a shapes for subsequent analysis (and to avoid any attempt to
function defined somewhere else). Shared data in this con- recursively inline a callback). This shape-building step can
text can be any one of external or static data, as well as be performed in three ways: dynamically performed at the
125
Figure 5: Synthesizing native calls
126
unable to compute even conditional results if, for example, 5.1 Implementation Status
arguments to a callback are fetched from storage. We have completed the W-Code conversion engine, and
During JNI use/def analysis, the results of GetObject- have added support to the TR inliner for inlining calls be-
Class, GetSuperClass, FindClass, GetMethodID and Get- tween native functions. We thus have a fully-functional Java
FieldID are treated as definitions and their uses are tracked. JIT compiler that can be substituted as a back-end for the
The results of these methods can normally be transformed various W-Code generating static front-ends. The correct-
into compile-time constants which are used instead of the ness of our implementation has been verified by successfully
usual constant pool indices. The use/def analysis also tracks compiling all of C benchmarks from SpecCPU2000 [32], as
the string arguments to FindClass, GetMethodID, and Get- well as standard C conformance tests. We have also com-
FieldID and by doing so, TR may positively resolve some piled these benchmark programs with native-side inlining
of these calls while a more naive implementation would be enabled and have observed the expected performance in-
unable to do so. creases.
Two significant changes made to the TR JIT to support
4.3.4 Transformations Using Use/Def Results W-Code-based languages include extending its data type set
Once JNI use/def analysis is complete, the procedure con- to include unsigned types (since it was originally designed
tinues by iterating over the list of identified callbacks and as Java JIT compiler), as well as modifying some of the
attempting to transform them to compile-time constant val- alias-analysis-dependent optimizations since aliasing in Java
ues or new TR-IL that is semantically equivalent and much is much simpler than in C. As noted in Section 4.1, alias
less expensive. Depending on the type of callback, the fol- information for the native code is explicit in the W-Code
lowing transformation outcomes are possible: IL, and is preserved during the transformation to TR-IL.
We have extended the TR inliner to include support for
• If all of the possible definitions reaching a GetObject- inlining a restricted set of callouts to native functions. The
Class are of the same class, the call is replaced by an mechanism for synthesizing calls to opaque native functions
appropriate constant. described in Section 4.2 has yet to be implemented, but we
expect to be able to leverage the existing Direct2JNI func-
• If all possible classes reaching a GetFieldID or Get- tionality for this purpose. At present, however, we can re-
MethodID are compatible and the string arguments cursively inline native functions defined in the same mod-
can be uniquely determined, the call is replaced by an ule and have verified the feasibility of transforming call-
appropriate constant. backs into cheaper direct references to JVM internals, by
applying the transformation operations to known callback
• If all possible field ids reaching a Get<type>Field or a functions. We have not yet built the generalized shape-
Put<type>Field are the same and all possible objects matching or the use/def analysis required to automatically
reaching the call are of compatible class types, the call detect and transform all callbacks. Non-transformable in-
is replaced by a new, simpler sequence of TR-IL. More lined callbacks, however, are successfully handled via calls
generally, if the offset of the data member from the to synthesized functions, and we have been successful in link-
beginning of the object is the same for all possible ing globally-declared data in native code to Java code.
types that can reach the call, then the same code can For the purposes of generating a proof-of-concept proto-
be used for all the objects, allowing the callback to be type, we have further restricted the features of Java that
“strength reduced”. Note that this form of IL defers we consider. We are currently ignoring string parameter ac-
throwing exceptions in accordance with the Java rules cess, reference creation, exception handling, the Reflection-
for executing native methods. related functions, monitors and the Invocation API. Al-
though these are important features that must be handled
• Similar transformations are performed for the various in a full implementation, we believe this is a matter of engi-
Call<type>Method callbacks by replacing the existing neering, and one that will not substantially alter the appli-
IL with new IL that makes direct calls to the function. cability of our native function inlining optimization.
Any of the identified callbacks that are not handled by the 5.2 Evaluation Methodology
steps above are treated as an ordinary call to an appropriate Native inlining is an optimization that interacts with the
VM service routine via a synthesized function. For any of performance dynamics of our JIT compiler, as well as with
these callbacks, if the use/def analysis produces known but the running Java program making native function calls via
inconclusive information, conditional logic may be inserted the JNI. As with any JIT optimization, the runtime cost
along with the appropriate IL that represents the semantics of performing the inlining and transformation must be bal-
of the callback being transformed. When the transformed anced against the expected benefit of removing overhead
callback is executed, appropriate behavior can be selected and exposing more IL to the JIT optimizer. Ultimately, we
based on actual values. Runtime safety checks on arguments believe the true power of this approach lies in the ability to
to callbacks can be performed as part of the callback trans- treat native and Java code together during JIT compilation,
formation process. particularly since we have the opportunity to eliminate pes-
simistic assumptions that the optimizer must make in the
presence of opaque calls. In this paper, however, we focus on
5. CURRENT STATUS & RESULTS the cost of converting native functions from W-Code IL into
In this section we report on the current implementation TR-IL, on the benefit of eliminating callout/return overhead
status of the design described in Section 4, and describe and transforming heavyweight callback operations into sub-
encouraging preliminary results. stantially cheaper operations. We also evaluate the runtime
127
Total Total Time per Without With
W-Code Time opcode Native Native
Benchmark Opcodes (ms) (µs) Microbenchmark Inlining Inlining Speedup
bzip2 15383 78.277 5.09 Test (ns) (ns) (X)
crafty 84693 466.952 5.51 instance
gap 336466 1797.185 5.34 0 args 423 0 ∞
gcc 133506 663.246 4.97 1 args 458 0 ∞
gzip 25469 139.263 5.47 3 args 490 0 ∞
mcf 5615 25.431 4.53 5 args 579 0 ∞
parser 48411 256.472 5.30 hash 535 97 5.5
perlbmk 279196 1596.122 5.72 static
twolf 105027 547.702 5.21 0 args 128 0 ∞
vortex 193413 1091.121 5.64 1 args 137 0 ∞
vpr 56756 310.426 5.46 3 args 138 0 ∞
5 args 143 0 ∞
Table 1: Cost of W-Code to TR-IL conversion for hash 176 96 1.8
SPECint 2000 C benchmarks
Table 2: Microbenchmark runtimes and improve-
ments with native inlining
savings due to our optimization for a microbenchmark that
performs data transfers between Java and C similar to those
required for JDBCTM [36]. on its size in W-Code opcodes. Further, note that the cost
We evaluate critical aspects of our proposed system using of conversion only needs to be paid once, when the function
microbenchmarks. All our timing measurements are per- is inlined, whereas the benefits of removing callout overhead
formed on an IBM 7038-6M2 with eight 1.4 GHz POWER4TM will be obtained on every subsequent use of the inlined code.
CPUs. In each case, we measure the time to perform 100
million calls and divide to obtain the reported per-call ex- 5.3.2 Impact of Inlining Callouts
ecution time. Unless otherwise stated, no additional opti- We implemented a series of microbenchmark tests that
mization was performed on the inlined native IL to highlight would show the overheads involved with performing call-
the impact of removing callout and callback overheads. outs. The results are shown in Table 2. Instance and static
To validate the applicability of native inlining and call- natives were implemented with varying numbers of param-
back transformations on real-world code, we profiled a run eters, and then JIT-compiled with and without the native
of SPEC JAppServer2004 using IBM Websphere R Applica- inlining optimization. For all static versions without native
tion Server 6.0. We found that 4.07% of all function calls inlining, Direct2JNI was used to create compiled glue code
made during the run were native calls to 71 unique native for the native call, as described in Section 3.1. The benefits
functions, accounting for roughly 23% of the running time. of Direct2JNI can be observed by contrasting the runtimes
Of these, 19 unique native functions were called at least 5000 of the static tests against the instance ones. In all but 0 args
times, and out of those, six were called at least 50,000 times. and hash, the native method bodies simply return one of the
A single native function, [Link](), was called more passed arguments. These tests show the incremental cost of
than 300,000 times. This suggests that the runtime cost of passing arguments to native functions. In most cases, the
inlining can be amortized over a large number of uses for resulting code after inlining performs so little work that it is
important native functions. If the native function is well- below the resolution of the timers (reported as 0 in Table 2).
understood by the compiler, semantic expansion [37] or a For native functions that contain real code, the benefits of
related inlining technique could be used to create a special- inlining alone will clearly depend on amount of time spent
case version. This approach, however, is less general than executing native code in the function. Examples could easily
our technique. be constructed showing speedups that range from effectively
5.3 Microbenchmark Results infinite (as for the 0 args function call) to effectively 0 (for
very long-running native functions). To see the benefits that
Using a series of small test cases, we evaluate our proto- might occur in one realistic use of native functions, we wrote
type in terms of the cost of IL conversion, and the benefits a native hash function3 . Inlining our instance hash function
of native inlining and callback transformation. gives a speedup of 5.5, while a static version gives a speedup
5.3.1 Cost of IL Conversion of 1.8.
These results show that for small native functions, re-
To evaluate the cost of converting from W-Code to TR- moving the overhead of callouts can be a significant benefit.
IL for C functions, we measured the time to convert the The primary motivation for inlining native code, however,
SPECint 2000 C benchmarks (eon is omitted because it is is to create the opportunity to transform the much more
in C++). Table 1 shows the results. We report the total expensive callbacks. We now consider the effect of these
number of opcodes converted, the total time for the conver- transformations.
sion, and the average time per opcode for each benchmark.
Overall, we find that the cost per opcode converted is small 5.3.3 Impact of Transforming Callbacks
(averaging just 5.3 microseconds), and reasonably similar
We also implemented a series of microbenchmark tests
across benchmarks. These results are encouraging, as they
suggest a simple heuristic should be able to estimate the 3
we based this hash function on Wang’s 32-bit mix function
cost of converting a given native function at runtime based at [Link] Ttwang/tech/[Link]
128
Without With 6. RELATED WORK
Native Native
This section describes previous research on language in-
Microbenchmark Inlining Inlining Speedup
teroperability, as well as work on optimizing Java native
Test (ns) (ns) (X)
functions and the codes they call.
instance Examples of language interoperability frameworks
GIntField 2560 0 ∞ that operate across languages, processes and machine bound-
SIntField 2310 0 ∞ aries include CORBA [30], Remote Procedure Calls [7] and
CVoidMethod 2630 204 12.9 the Component Object Model [23]. These frameworks use
static interface definition languages (IDLs) to specify common types,
GStaticIntField 2190 0 ∞ and depend on proxy stubs to help clients translate between
SStaticIntField 2140 0 ∞ machine architectures, execution models and programming
CStaticVoidMethod 2520 214 11.8 languages. A more recent advance in language interoper-
GArrayLength 5640 60 93.4 ability is Microsoft .NET [15]. In order for .NET programs
to be interoperable, programs must adhere to the Common
Table 3: Microbenchmark runtimes and improve- Language Specification (CLS), a subset of the Common Lan-
ments with native inlining and callback transforma- guage Runtime (CLR) environment. The CLS most notice-
tions ably sets restrictions on the data types that can be used,
confines interoperability to .NET-labeled languages, and ex-
Without Native With Native cludes C or C++.
Array Inlining Inlining Speedup Programmer-based optimizations put the onus on
Length (ns) (ns) (X) the application programmer to practice efficient coding tech-
1 586 2.4 244.1 niques when writing native code that uses the JNI. A former
10 597 20.7 28.9 IBM developerWorks R article [19] advised batching native
100 1010 85.5 11.8 calls and passing as much data as possible per native call, as
1000 4540 600 7.6 well as a number of other recommendations to amortize over-
head. Although this article is no longer available through
Table 4: Moving data from Java to C - runtimes developerWorks, the recommended JNI programming prac-
and improvements with native inlining and callback tices are still valid. Similarly, the JNI specification [22] pro-
transformations vides a set of “critical” functions that may return direct ref-
erences to JVM data and objects and suggests ways to avoid
making JNI callbacks, including caching field and method
that would show the overheads involved with performing IDs during static initialization of classes. By reducing the
callbacks in native code, and the benefits of transforming overhead of the JNI automatically, our approach obviates
these callbacks in inlined natives. Instance and static na- these programming practices, removing the added burden
tive functions consisting of a series of callbacks were JIT- from the application programmer.
compiled with and without the native inlining and trans- Restricting functionality that can be offered in native
formation optimization. For example, the CVoidMethod code is another approach to reduce overhead and minimize
native code calls GetObjectClass (which is transformed to a the dependence on JNI callbacks. For example, the Intel
compile-time constant), GetMethodID, (also a compile-time ORP [10] supports a “direct call” mechanism that bypasses
constant) and finally, CallVoidMethod (which is transformed the construction of special wrapper codes that would other-
to a non-JNI-dependent virtual function call using the con- wise be found preceding native call frames on the call stack.
stants from the previous two transformations). The GAr- The speedup that results from not having to perform main-
rayLength test creates an array of characters using NewChar- tenance work in the wrappers comes at the expense of not
Array and returns its length using GetArrayLength. The being able to unwind the stack. Therefore, direct calls can
native code in this last example is transformed into the only be used for methods that are guaranteed not to require
equivalent TR-IL. The results are shown in Table 3. Be- garbage collection, exception handling, synchronization or
cause callbacks are more expensive than callouts, we see that any type of security support. Bacon [3] has implemented
the benefit of transforming them is correspondingly greater, a JVM-specific JNI “trap-door”, simplifying reference man-
with a minimum achieved speedup of nearly 12X in our test agement for garbage collection, based on the observation
cases. that his native code only accesses parameters and never per-
The final set of microbenchmark tests that we present deal forms any JNI callbacks. While these strategies can improve
with passing integer array data from Java to native code, performance in certain circumstances, they are not a general
an operation that is commonplace in native codes used by solution and cannot be used for most existing JNI code.
JDBC drivers. In these experiments, a single callback is used Proprietary native interfaces that are used by var-
to obtain the entire array, similar to the “coarse-grained” ious VMs take advantage of knowing the internals of the
strategy of JNIbench [1]. Table 4 displays the speedups ob- VM, and therefore help mitigate the overheads of native
tained by transforming inlined GetIntArrayRegion callbacks calls by tightly coupling them to the VM. Examples include
for each of the varying array lengths, which vary from a fac- the PERC Native Interface for the PERC VM [26], the Jcc
tor of 244 for a single element array, to a factor of 7.6 for optimizing compiler [34], Microsoft’s now-supplanted Raw
a 1000-element array. As expected, these speedups decrease Native Interface (RNI) [11], and according to [25], the orig-
for larger array sizes because the overhead in performing the inal but now deprecated Native Method Interface (NMI).
callout and callback shrinks relative to the actual work done All of these approaches closely couple the interface with the
in transferring the array. specific virtual machine, and thus seriously restrict porta-
129
bility. The JNI, in contrast, is a cleaner and more portable Although a JIT compiler is unlikely to be able to compete
solution because all JVM internals are represented in an with a static native code optimizer, the W-Code IL stored
opaque manner and can only be accessed via JNI callbacks. alongside our native binaries is the output of a sophisticated
This, ironically, lies at the heart of JNI-related overheads. interprocedural optimizer and loop transformer. This pro-
Our work is independent of the JVM being used and our vides the TR JIT with some of the benefits of static analysis
technique can be utilized by those who want to support it. that could not be contained in the compile-time budget of
A different approach involves extending the JVM to sup- a dynamic compiler. We also expect that some of the run-
port features for which native functions are commonly used. time information unavailable to static optimizers will help
One example is incorporating unmanaged memory into further improve the quality of the inlined native code. Cur-
the JVM. The provision of high-speed access to unmanaged rently, we aren’t overly concerned with modifying the TR
memory can be used to implement shared memory segments, JIT inliner’s heuristics, except for some fine-tuning that rec-
memory-mapped files, communication and I/O buffers, and ognizes a number of differences between Java and non-Java
even memory-mapped hardware devices. Jaguar [35] imple- functions, including their sizes, variable length parameter
ments Bytecode-to-assembly code mappings in a JIT com- lists and parameters whose addresses are taken. Studying
piler to generate inlined assembly code for limited sequences suitable heuristics for native inlining is a subject for future
of machine code. The main use of the mappings is to map work.
object fields to memory outside the managed Java heap. The
benefit of this approach is the near-C performance obtained
for various latency and bandwidth simulations. However,
8. CONCLUSION
there are a number of limitations with this approach, in- We have shown that native function inlining can greatly
cluding the inability to recognize and map long, complex reduce the overheads normally associated with Java native
sequences of Bytecodes, as well as the inability to apply function calls and callbacks. One key component of our
mappings to virtual method invocations, since code map- strategy is an IL conversion engine that allows code written
pings can’t handle runtime class loading. Buffers in the in low-level languages to be expressed in the same IL used
Java new I/O libraries [18] are also allocated outside the by the TR JIT for Java code. This facilitates the inlining
garbage-collected heap, and can be accessed by the JVM of native function calls defined in native libraries that in-
without having to perform any time-consuming copy oper- clude the IL of a static optimizing compiler, and the further
ations. Our technique is orthogonal to the work on unman- transformation of callbacks. One of the key benefits of our
aged memory. strategy is that the full JNI API is maintained while giving
Optimizations that target native functions and the JNI the performance characteristics of direct access to Java ob-
specifically include IBM’s enhancements to the Java 2 Plat- jects from C. Our goal for this technique is both to improve
form mentioned in [19] and inlining of helper code that the performance of applications that make extensive use of
sets up JNI stack frames as mentioned in [10]. IBM also the JNI, and to remove the need for programming practices
reuses existing Java stack frames to reduce the native stack designed to circumvent the currently-heavy penalty of using
frame setup overhead. Andrews’ [1] suggestions include na- the JNI.
tive memory mirroring as well as provisioning the JNI for
lightweight calls. No known implementations of native func- 9. ACKNOWLEDGMENTS
tion inlining exist, but have been referred to by Andrews [1] This research is supported by grants from the IBM Centre
and by Liang [22] as a powerful yet difficult-to-implement for Advanced Studies, CITO, and NSERC. It would also
optimization. Our solution demonstrates that native func- not have been possible without the support of the TR JIT
tion inlining is feasible with a JIT compiler, and that the compiler team at the IBM Toronto Software Lab.
benefits of removing overhead alone may make it worth-
while. Further, it enables more aggressive optimizations, developerWorks, IBM, POWER4, and WebSphere are trade-
similar to traditional inlining techniques. marks or registered trademarks of International Business Ma-
chines Corporation in the United States, other countries, or both.
Java, JDBC, and all Java-based trademarks are trademarks of
7. FUTURE WORK Sun Microsystems, Inc. in the United States, other countries, or
There are some issues that make our proof of concept both.
harder to generalize. Native callsite polymorphism, includ- Microsoft is a trademark of Microsoft Corporation in the United
States, other countries, or both.
ing overriding and overloading of natives, as well as syn- Intel is a registered trademark of Intel Corporation in the United
chronized natives are outside the scope of our immediate States, other countries, or both.
work. Other engineering issues we currently ignore include Other company, product, and service names may be trademarks
inlining native code located in modules external to the one or service marks of others.
containing the original native callout.
The pattern matching basis of our callback transforma- 10. REFERENCES
tion algorithm may be too strict to handle all types of native [1] Jack Andrews. Interfacing Java with native code:
code, and we currently only analyze a small subset of JNI Performance limits. ITtoolbox for Java Technologies
callbacks. We are working on completing the implementa- Knowledge Base web site, Peer Publishing section. http:
tion of the design described in Section 4, which will allow //[Link]/documents/[Link]?i=780#,
us to explore questions of when the native inlining strat- 2000. Also available at [Link]
[2] Andrew Ayers, Stuart de Jong, John Peyton, and Richard
egy should be performed, and when the existing native call
Schooler. Scalable cross-module optimization. In PLDI ’98:
should be left alone. We also plan to evaluate the impact Proceedings of the ACM SIGPLAN 1998 conference on
of exposing native code to the TR optimizer, and study the Programming language design and implementation, pages
end-to-end effect on realistic benchmarks. 301–312. ACM Press, 1998.
130
[3] David F. Bacon. JaLA: A Java package for linear algebra. [21] Chris Lattner and Vikram Adve. LLVM: A compilation
Presented at the Computer Science Division, University of framework for lifelong program analysis & transformation.
California, Berkeley, 1998. IBM T.J. Watson Research In Proceedings of the 2004 International Symposium on
Center. Code Generation and Optimization, pages 75–87, San Jose,
[4] Mark Baker, Bryan Carpenter, Geoffrey Fox, Sung California, March 20–24 2004.
Hoon Ko, and Xinying Li. mpiJava: A Java interface to [22] Sheng Liang. The Java Native Interface. Programmer’s
MPI. In Proceedings of the First UK Workshop, Java for Guide and Specification. Addison-Wesley, 1999.
High Performance Network Computing at EuroPar, [23] Microsoft Inc. Com: Component object model technologies.
Southampton, UK, September 1998. [Link]
[5] Paolo Bellavista, Antonio Corradi, and Cesare Stefanelli. [24] Michael Lazar Milvich. JavaCAVE: A 3D immersive
How to Monitor and Control Resource Usage in Mobile environment in Java. Master’s thesis, Montana State
Agent Systems. In Proceedings of the Third IEEE University, July 13 2004.
International Symposium on Distributed Objects and [25] Paul M. Murray, Todd Smith, Suresh Srinivas, and Mattias
Applications, pages 65–75, Rome, Italy, September 17–20 Jacob. Performance issues for multi-language Java
2001. applications. In Proceedings of the 15 International Parallel
[6] Aart J. C. Bik and Dennis Gannon. A Note on Native and Distributed Processing Symposium 2000 Workshops,
Level 1 BLAS in Java. Concurrency: Practice and volume 1800 of Lecture Notes in Computer Science, pages
Experience, 9(11):1091–1099, 1997. 544–551, Cancun, Mexico, May 1–5 2000. Springer.
[7] Andrew D. Birrell and Bruce Jay Nelson. Implementing [26] NewMonics Inc. Best practices for native code integration
remote procedure calls. ACM Transactions on Computer with perc. [Link]
Systems, 2(1):39–59, 1984. [Link], February 26 2003.
[8] Per Bothner. Java/C++ integration - writing native Java [27] Steve Northover. SWT: The Standard Widget Toolkit, Part
methods in natural C++. 1: Implementation Strategy for JavaTM Natives.
[Link] [Link]
November 1997. Article-SWT-Design-1/[Link], March 2001.
[9] Fabian Breg and Constantine D. Polychronopoulos. Java [28] Hirotaka Ogawa, Kouya Shimura, Satoshi Matsuoka,
virtual machine support for object serialization. In Fuyuhiko Maruyama, Yukihiko Sohda, and Yasunori
Proceedings of the 2001 Joint ACM-ISCOPE Conference Kimura. OpenJIT: An Open-Ended, Reflective JIT
on Java Grande, pages 173–180, Palo Alto, California, June Compiler Framework For Java. In Proceedings of the 14th
2–4 2001. European Conference on Object-Oriented Programming,
[10] Michal Cierniak, Marsha Eng, Neal Glew, Brian Lewis, and volume 1850 of Lecture Notes in Computer Science, pages
James Stichnoth. The Open Runtime Platform: A Flexible 362–387, Sophia Antipolis and Cannes, France, June 12–16
High-Performance Managed Runtime Environment. Intel 2000. Springer.
Technology Journal, 7(1), February 2003. [29] Vladimir Roubtsov. Profiling cpu usage from within a Java
[11] Bruce Eckel. Thinking in Java. Prentice-Hall, 1st edition, application. [Link]
1998. javaqa/2002-11/[Link], November 2002.
[12] Robert Fitzgerald, Todd B. Knoblock, Erik Ruf, Bjarne [30] Todd Scallan. a corba primer.
Steensgard, and David Tarditi. Marmot: An optimizing [Link]
compiler for Java. Technical Report MSN-TR-99-33, June 3 2002.
Microsoft Inc., June 16 1999. [31] Davanum Srinivas. Java tip 86: Support native rendering in
[13] Michael Franz and Thomas Kistler. Slim binaries. jdk 1.3. [Link]
Communications of the ACM, 40(12):87–94, December [Link].
1997. [32] Standard Performance Evaluation Corporation. SPEC
[14] Vladimira Getov, Susan Flynn Hummel, and Sava CPU2000 V1.2. [Link]
Mintchev. High-performance parallel programming in Java: [33] Vaidy Sunderam and Dawid Kurzyniec. Efficient
exploiting native libraries. Concurrency: Practice and cooperation between Java and native codes – JNI
Experience, 10(11–13):863–872, 1998. performance benchmark. In Proceedings of the 2001
[15] Andrew D. Gordon and Don Syme. Typing a International Conference on Parallel and Distributed
multi-language intermediate code. ACM SIGPLAN Processing Techniques and Applications, Las Vegas,
Notices, 36(3):248–260, March 2001. Nevada, June 25–28 2001.
[16] James Gosling, Bill Joy, and Guy Steele. The Java [34] Ronald Veldema. Jcc, a native Java compiler. Master’s
Language Specification. Addison-Wesley, 1996. thesis, Vrije Universiteit, Amsterdam, August 1998.
[17] Bjørn-Ove Heimsund. Native Numerical Interface (NNI). [35] Matt Welsh and David Culler. Jaguar: Enabling efficient
[Link] November communication and I/O in Java. Concurrency: Practice
2004. and Experience, 12(7):519–538, May 2000.
[18] Ron Hitchens. Java NIO. O’Reilly and Associates, Inc., [36] Seth White, Maydene Fisher, Rick Cattell, Graham
August 2002. Hamilton, and Mark Hapner. JDBCTM API Tutorial and
[19] IBM Corporation. IBM rewrites the book on Java Reference: Universal Data Access for the JavaTM 2
performance. http: Platform (2nd Edition). Pearson Education, June 1999.
//[Link]/java/j2/[Link]. [37] Peng Wu, Samuel P. Midkiff, Jose E. Moreira, and Manish
[20] IBM Corporation. XL FORTRAN: Eight ways to boost Gupta. Efficient support for complex numbers in java. In
performance. White Paper, 2000. Proceedings of the ACM 1999 Java Grande Conference,
pages 109–118, San Francisco, California, June 1999.
131