Merkle-Damgård Construction Method and Alternatives: A Review
Merkle-Damgård Construction Method and Alternatives: A Review
UDC 004.056.55
Survey Paper
Abstract
Cryptographic hash function is an important cryptographic tool in the field of
information security. Design of most widely used hash functions such as MD5 and
SHA-1 is based on the iterations of compression function by Merkle-Damgård
construction method with constant initialization vector. Merkle-Damgård construction
showed that the security of hash function depends on the security of the compression
function. Several attacks on Merkle-Damgård construction based hash functions
motivated researchers to propose different cryptographic constructions to enhance the
security of hash functions against the differential and generic attacks. Cryptographic
community had been looking for replacements for these weak hash functions and they
have proposed new hash functions based on different variants of Merkle-Damgård
construction. As a result of an open competition NIST announced Keccak as a SHA-3
standard. This paper provides a review of cryptographic hash function, its security
requirements and different design methods of compression function.
1. Introduction
Cryptographic hash function is a one-way and compression function that converts an
arbitrary length message to a fixed length hash value. This hash value of a message
is also known as the fingerprint of the message. Any small change or modification in
the input data causes the drastic change in the hash value. Cryptographic hash
function is widely used in security applications and protocols [1]. Hash functions are
targeted heavily by cryptanalysts as they are a fundamental building block for many
security applications. Cryptographic hash function ensures the integrity and
authentication in the communication. There are various applications of
cryptographic hash function such as pseudo-random string generation, digital
signature and MAC.
The basic operation of cryptographic hash function has been shown in Figure 1.
A cryptographic hash function is like a deterministic and computationally efficient
random function. Cryptographic hash function has to satisfy requirements of
onewayness and collision resistance. Onewayness means that the method to
283
TIWARI MERKLE-DAMGÅRD CONSTRUCTION METHOD...
input is arbitrary length of any binary string, and the output is n bits of binary string.
We usually call n as the size of hash value. A hash function must satisfy the
following properties;
Compression: h maps an input M of an arbitrary bit length (up to a predefined very
long maximum length) to an output of a fixed bit length n.
Ease of computation: for a given input M and a hash function h, the process of
computing h(M) should be easy and fast.
(1) Preimage resistance: it is computationally infeasible to find any input which
hashes to any pre-specified output i.e., given a hash value H, it is
computationally infeasible to find an input M such that h(M) H.
(2) Second preimage resistance: it is computationally infeasible to find any
second input which has the same output as any specified input. That is,
given an input M, it is computationally infeasible to find another input
Msuch that h(M) h(M) and M M. This is also known as weak
collision resistance.
(3) Collision resistance: it is computationally infeasible to find two different
inputs with the same output. That is, it is computationally infeasible to find a
284
JOURNAL OF INFORMATION AND ORGANIZATIONAL SCIENCES
pair of inputs, M and M, such that h(M) h(M) and M M. This is also
known as strong collision resistance.
(4) Near-collision resistance: it is computationally difficult to find any two
different inputs M and M, that have a low Hamming weight between their
hash values, i.e., h(M) differs from h(M)by a few number of bits.
(5) Partial preimage resistance: given a hash value, it is computationally
difficult to recover any part of the message.
(6) Non-correlation: the input bits of an input M should not be correlated to the
output bits of h(M) .
(7) Random behaviour: hash function should have random behaviour. That is,
given a particular input M it should be infeasible to predict any output bits
of h(M) without actually applying the function h.
(8) Deterministic nature: hash function h should be deterministic, i.e. given a
particular input M, the function always computes the same output h(M) .
285
TIWARI MERKLE-DAMGÅRD CONSTRUCTION METHOD...
OWHF. CRHF usually deals with longer length hash values. Other then these
functions universal one way hash function (UOWHF) also exist [1], [13].
Paper discussed basic hash function design and MD construction in section 2.
Section 3 gives iterative processing of the messages by different alternative hash
construction methods. In this section weaknesses and security of compression
function have also been highlighted. Paper is concluded in section 4.
The compression function has the same security requirements that a hash function
but takes fixed length inputs. The domain extension algorithm defines how to use
the compression function in order to handle arbitrary length inputs. Almost all hash
functions are iterative processes which hash inputs of arbitrary length by processing
successive fixed-size blocks of input. In this section, we will discuss some popular
known iterative hashing constructions. Common iterative structure is shown in the
Figure 2.
286
JOURNAL OF INFORMATION AND ORGANIZATIONAL SCIENCES
takes a b-bit message block and an n-bit chaining value. Let h : 0,1 0,1 be a
* n
287
TIWARI MERKLE-DAMGÅRD CONSTRUCTION METHOD...
value IV. The block diagram of the iterative hash function using the compression
function is shown in the Figure 3. The computation of the hash value is dependent
on the chaining variable. At the start of hashing, this chaining variable has a fixed
initial value which is specified as part of the algorithm.
This process continues recursively, with the chaining variable being updated under
the action of different part of the message, until the entire message has been used.
The final value of the chaining variable is then output as the hash value
corresponding to that message. One of its distinctive features is that it promotes the
collision resistance and preimage resistance of the compression function to the full
hash function: for instance, a collision on the compression function can be deduced
efficiently from a collision on the full hash function. The inclusion of the length at
the end of the message is important for this situation, and is also important for
preventing a number of attacks, including long-message attacks.
Merkle-Damgård construction proves that the security of hash function relies on
the security of the compression function. Thus, in order to build a collision resistant
hash function, it is sufficient to design a collision resistant compression function.
Recent results, however, highlight some intrinsic limitations of the MD approach
[27]. This includes being vulnerable to multicollision attacks [10], long second
288
JOURNAL OF INFORMATION AND ORGANIZATIONAL SCIENCES
preimages attacks [11], and herding attack [12]. Figure 4 shows detailed view of MD
construction.
However, the Damgård tree construction is not practical since the size of the binary
tree grows with the length of the message. Figure 5 illustrates a typical tree based
hashing construction. Damgård tree construction was later optimized by Sarkar and
Scellenberg [14]. Sarkar and Scellenberg construction (SS construction) was a
parallel version of MD construction. The main difference between SS construction
and previous constructions is that authors consider the number of available
processors to be fixed while the length of the message can be arbitrarily long.
Thus SS construction considered a fixed processor tree and used it to hash
arbitrarily long messages. Each processor simply computes the base hash function.
Similarly, Carter and Wegman [15] used tree hashing techniques to build universal
hash functions. This was followed up by Naor and Yung [16] and Bellare and
Rogaway [17] in the context of UOWHFs (Universal One Way Hash Functions). In
[18] Bellare and Micciancio proposed the randomize-then-combine paradigm, where
the message is split into blocks, each block is processed via randomizing function
289
TIWARI MERKLE-DAMGÅRD CONSTRUCTION METHOD...
(derived from some standard hash function) and finally combined by an operation
such as XOR. Although this structure was originally proposed to build incremental
functions, it can be thought of as a 2- level tree and can still be parallelized since the
randomization process of the individual blocks are independent. Tree-based
constructions are slightly less popular than the iterative ones.
Basically, sponge hashing proceeds in two phases, the absorbing phase and the
squeezing phase. The sponge operates on a fixed length state s 0,1 composed
r c
absorbing phase, the message is divided into r-bit blocks (padded if necessary) and
each block is XORed with the r part of s (initially, s 0 ), f then iteratively
r c
processes s until all blocks are exhausted. In the squeezing phase, the state continues
to be transformed or permuted by f but this time the r parts of the states are returned
at every iterations as output blocks. Since the sponge construction supports variable
length output, the user chooses the length of the final hash value which determines
how many of the returned blocks in the squeezing phase need to be returned.
Optionally, between two phases, some number of blank rounds can be applied. In
290
JOURNAL OF INFORMATION AND ORGANIZATIONAL SCIENCES
the blank rounds, there is no input to or output from the state. Only, f is applied to
the state s. If f is expressed as a random function, the construction is called a T-
sponge, otherwise if it is expressed as a permutation then the construction is called a
P-sponge. The security of a sponge construction depends on its capacity c, hash size
n and function f. For P-sponge construction the complexity of a collision is
Min (2 c 2 , 2 n 2 ) and complexity of preimage and second preimage is Min (2 c 2 , 2 n ) .
Collision complexity of T-sponge construction is equal to the collision complexity
of P-sponge construction. Finding a preimage costs Min (2 c , 2 n ) and finding a second
preimage costs Min(2c N ,2n ) for a T-sponge, where N is the length of the original
message. Figure 6 illustrates the sponge construction.
Hash functions such as Keccak [20] and PHOTON [21] are based on the
sponge construction. Keccak has recently been selected as the winner of SHA-3
competition. Although still considered an iterative construction, the sponge is
completely different from the Merkle-Damgård construction. When iterated hash
functions are considered, there always exist inner collisions which can be defined as
if two message pair M1and M2give the same chaining value, then concatenation of
M1and M2with collide suffix M*collide. In the sponge function construction, there
also exist inner collisions and this is the only weaknesses of sponge functions so far.
291
TIWARI MERKLE-DAMGÅRD CONSTRUCTION METHOD...
compression functions:
f : 0,1 0,1 0,1 and f : 0,1 0,1 . Wide pipe hash is computed
w m w w n
as:
H i f ( H i 1 , M i ), i 1t ,
h( M ) f ( H t )
(2)
Wide-pipe construction is shown in the Figure 7.
On the other hand, the double pipe design maintains twice the hash size using the
w 2 n compression function in parallel to process each message block. Using one
compression function f : 0,1 0,1 0,1 , with m n and two distinct
n nm n
random initial values H 0 H 0 0,1 double pipe hash is computed as:
n
H i f ( H i1 , H i1 M
i ), i 1 t 1
H i f ( H i1 , H i1 M
i ), i 1 t 1
h( M ) f ( H t1 , H t1 M t )
(3)
292
JOURNAL OF INFORMATION AND ORGANIZATIONAL SCIENCES
become larger than the size of the final hash value, would significantly improve the
security of the hash function. This modification clearly thwarts the extension attack
since in the wide and double pipe construction the final hash value is truncated, so in
order to append an extension, the unknown discarded bits have to be guessed, which
is clearly difficult if the number of the discarded bits is non-trivial. Furthermore, by
increasing the size of the internal state, finding collisions for the compression
function becomes harder, which complicates the other generic attacks. An obvious
drawback of the wide and double pipe construction, however, is a degraded
efficiency as the compression function now has larger input and output while
keeping the hashing rate constant (the size of the compression function input
corresponding to a message block is fixed) since the chaining variable input is
increased. Also, adapting existing hash functions for the wide and double pipe
construction may be difficult since it might be the only reasonable way to increase
the internal state is to use multiple compression function calls in parallel for every
iteration. Recently, Yasuda [23] adopted a slightly modified variant of the double
pipe construction and proved its unforgeability beyond the birthday barrier.
3.4. 3C Construction
The 3C construction is the simplest variant of the MD construction that one can
obtain to improve its security against multi block collision attack [24]. The 3C hash
function processes the intermediate chaining values of the MD construction by
maintaining a second internal chaining variable containing a value produced by
repeatedly XORing the chaining variables while hashing a message; this variable is
then processed in an extra finalisation call to the compression function. There are
two chains in 3C construction: the accumulation chain and cascading chain. The
accumulation chain and the compression function have an accumulator XOR
function that works iteratively in the cascade chain, similarly to the MD
construction. The processing in the 3C divides the message into t-blocks with IV0
representing the initial value. ai and ci are the chaining variables in the accumulation
chain and cascade chain. The compression functions are executed three times for
each block: the processing data block, padding block and forming the block Z in the
accumulation chain. The 3C is as secure as the MD construction.
The 3C hash is computed as:
c0 IV0 ,
ci f ci 1 , M i , i 1t ,
a1 c1
ai ai 1 ci , i 2 t , Z at ,
h( M ) g ( Z , ct ).
(4)
To increase the security level of 3C, 3C+ design has been proposed. In the 3C+ hash
construction, there is an additional chain called the final chain. The final chain is
293
TIWARI MERKLE-DAMGÅRD CONSTRUCTION METHOD...
added to the cascade and accumulation chains of the 3C hash construction. The final
compression function g at the last block takes the series of the result of the
accumulation and final chains after padding. Due to this enhancement, the security
level of 3C+ is higher than that of 3C and MD construction. 3C+ uses extra memory,
but makes finding multi-block collisions more difficult. However, the 3C and 3C+
structures are slower because the processors have to process data sequentially, where
every block takes its input from the previous block, causing sequential delay.
However, in [25], it was shown that both 3C and 3C+ are indeed susceptible for
multi-block attack. The designers of 3C claimed that while it is susceptible to the
multi-collision attack, it resists the long messages 2nd pre-image and herding attacks.
However, it was shown in [26] that 3C is also indeed susceptible to both the second
preimage and herding attacks.
3.5. The Prefix Free, Chop Constructions, NMAC and HMAC Constructions
Coron et al. proposed, prefix-free, NMAC, and HMAC constructions as secure
variants for the MD construction [28]. Later it is found that even though these
constructions are indifferentiable from RO, they are not collision resistant. The
prefix-free construction does not modify the Merkle-Damgård construction, instead
it modifies the padding algorithm to make sure that the message is prefix free. One
way to do this is by prepending or appending the length of the whole message to
every message block. PFMD construction uses a padding function g which ensures
that for any two messages M , M with M M , g (M ) cannot be a prefix of g (M ) .
Let N is the length of the message M. Three variants of PFMD are:
Varient1: PFg1
y0 IV ,
M ( M1 ,, M t ), M i b
g1 ( M ) ( N M1 M t ),
y f ( y0 , g1 ( M ))
(5)
Varient2: PFg 2
y0 IV ,
M ( M1 , , M t ), M i b 1
g 2 ( M ) ((0 M1 ) (0 M 2 ) (0 M t 1 ) (1 M t )),
y f ( y0 , g 2 ( M ))
(6)
294
JOURNAL OF INFORMATION AND ORGANIZATIONAL SCIENCES
Varient3: PFg 3
y0 IV ,
M (M1 ,, M t ), M i b
( yi 1 , M i , N , i )), i 1,, t
yi f
(7)
The Chop construction is an n bit MD hash where r out of n bits of the hash value
are chopped, thus producing an (n−r)-bit hash value. The chop construction
basically removes a non-trivial number of bits from the final hash value. This, while
it solves the indifferentiability issue, unfortunately lowers the security bounds of the
hash function.
In NMAC, an independent function g is applied to the output of the last
application of the compression function, while HMAC is a special case of the
NMAC in which an extra compression function call is introduced. The HMAC hash
construction hashes a message by applying the same f function twice, using the same
IV. The NMAC and HMAC are computed as:
NMAC:
y0 IV ,
yi f ( yi 1 , M i ), i 1, , t
yl g ( yt )
(8)
HMAC:
M 0 0b ,
y0 f ( IV , M 0 )
yi f ( yi 1 , M i ), i 1, , t
bn
yl f ( yt 0 , IV )
(9)
295
TIWARI MERKLE-DAMGÅRD CONSTRUCTION METHOD...
vector IV and the final key is XORed with the final intermediate chaining value,
while the final hash result is left unmodified.
y0 IV0 ,
yi f ( yi 1 , M
i ), i 1, , t 1
yt f ( yt 1 M t , IV1 )
(10)
MDP construction is shown in the Figure 9. The authors proved that the collision
resistance of MDP follows trivially from the collision resistance of the Merkle-
Damgård construction as the former introduces minimal changes to the latter. The
296
JOURNAL OF INFORMATION AND ORGANIZATIONAL SCIENCES
authors also discussed the security of possible simple MAC constructions based on
MDP. However, although with such a simple modification, the authors succeeded in
proving a significant security gain, MDP seems to be able to thwart only the
extension attack, but not other Merkle-Damgård generic attacks. Also, recently it
was shown that MDP is neither pre-image nor second preimage resistant [31].
y0 IV0 ,
yi f ( yi 1 , M
i ), i 1, , t 1
y f ( ( yt 1 ), M t )
t
(11)
H 0 IV ,
H i f ( H i 1 , M i ), i 1, , t
H 0' H t
'
H i g ( H i 1 , M t i 1 ), i 1, , t
'
(12)
297
TIWARI MERKLE-DAMGÅRD CONSTRUCTION METHOD...
using M and sign. It aims the provision of security guarantees even when the
compression function is compromised with respect to collision security. It was
formally showed that just finding collisions on the compression function is not
sufficient in order to break the resultant signatures: instead, the attacker needs to
solve a much harder cryptanalytical problem, closer to finding second preimages.
The authors claim that randomized hashing will strengthen any hash function, even
the weakest ones.
y0 f ( IV , s )
yi f ( yi 1 , M i
s ), i 1, , t
h( M ) yt
(13)
In the dither hash function, every call to the compression function f has the three
inputs: the dithering sequence D D1 ,, Dt which depends on the iteration, the
298
JOURNAL OF INFORMATION AND ORGANIZATIONAL SCIENCES
chaining value, and the next message block from M M1 ,, M t and generates hash
as follows:
H 0 IV ,
H i f ( H i 1 , M i , Di ), i 1, , t
h( M ) H t
(14)
Figure 10 shows the Dither construction. The dither value can be selected in many
ways: one of the ways by following the suggestion of Kelsey and Schneier the dither
value can be selected as the index, Di i . This approach is called the dithering by
counter but this approach requires that compression function accept an arbitrary
large input. Another suggestion for selecting the dither value can be a sequence of
alternative 0’s and 1’s.
A pseudorandom sequence can also be used as a dither value. This provides
protection against message block repetition. In his proposal Rivest suggested the use
the infinite abelian square-free sequence. The abelian square-free sequence is an
aperiodic sequence over a finite alphabet with the property that no sub-word is
repeated. However, the method proposed for integrating the dither value into
concrete hash functions is inefficient, in the sense that it increases the number of
calls to the compression function. No indifferentiability result is known for the
Dither hash function.
(15)
The padding in HAIFA is very similar to the padding of Merkle-Damgård
[Link], the padding is done by appending a single '1' bit followed by
as many '0' bit as needed to complete an b-bit block after the message length and the
digest size are appended. Then input M subsequently divided into t blocks
299
TIWARI MERKLE-DAMGÅRD CONSTRUCTION METHOD...
3.13. BCM
The backwards chaining mode was proposed by Andreeva and Preneel. It uses three
keys k1 , k2 , k3 and of fixed length (b + 2n) bits, where k2 b and k
1 k
3 n where
n is the state and b is the block size. It XORs the key k1 and the most significant n
bits of block M 2 with the fixed initial chaining variable IV. The message block M1
together with the resulting value from the XOR computation form the input to the
first application of f. In the iteration the message block M i and the chaining variable
Hi 1 in-line are XORed with the most significant n bits of the next-in-line message
block M i 1 and form the inputs to the ith compression function f. The one but last
block M t 1 is interpreted differently than the rest of the message blocks. Here the
difference is that the least significant n bits of M t 1 are XORed with the key k1 , the
chaining variable H t 2 is XORed with the first significant bits of k2 and M t . The
final input to the last compression function is provided by the last message block
M t and the chaining variable Ht 1 XORed with keys k2 and k3 , respectively.
300
JOURNAL OF INFORMATION AND ORGANIZATIONAL SCIENCES
H 0 IV ,
H i f ( H i 1 , M i ,
k ), i 1, , t 1
H t f ( H t 1 , M t , k )
(17)
H 0 IV ,
f ( H i 1 k mask , M i ), i
Hi 1, , t
(18)
H 0 IV1 ,
H i f ( H i 1 , M
i ), i 1, , t 1
H t f ( IV2 H t 1 , M t )
(19)
4. Conclusion
A cryptographic hash function plays a vital role in many security applications and
protocols such as digital signatures and authentication schemes. Among several
security requirements collision resistance is an important property of a cryptographic
hash function. The security of hash function depends on the collision resistance
property of the underlying compression functions. Merkle-Damgård construction
method failed to preserve this important security property. In recent years, it has
301
TIWARI MERKLE-DAMGÅRD CONSTRUCTION METHOD...
been shown that hash functions based on weak Merkle-Damgård construction are
vulnerable to different attacks. As a result researchers have given different
construction methods to design the hash functions such as Haifa, Dither and Tree.
This paper reviews different popular alternative construction methods to Merkle-
Damgård method and also discusses how these methods have strengthened the weak
MD method. The use of hash functions based on Merkle-Damgård construction in
different security products, services, algorithms and protocols makes them
vulnerable to different cryptanalytic attacks.
References
[1] A.J. Menezes, P.C. vanOorschot, and S.A. Vanstone, “Handbook of
applied cryptography,” 1997.
[2] W. Diffie and M. Hellman, 1976, “New Directions in Cryptography,”
IEEE Transaction on Information Theory, 1976, vol. 22, pp. 644-654.
[3] R. C. Merkle, “One Way Hash Functions and DES,” Crypto’89, 1989,
LNCS, vol. 435, pp. 428-446.
[4] I. Damgård, “A Design Principle for Hash Functions,” Crypto’89, 1989,
LNCS, vol. 435, pp. 416-427.
[5] R. Rivest, “The MD4 Message Digest Algorithm,” Request for Comments
(RFC) 1320, Internet Engineering Task Force, 1992.
[Link]
[6] R. Rivest, “The MD5 Message Digest Algorithm,” Request for Comments
(RFC) 1321, Internet Engineering Task Force, 1992.
[7] NIST, “Secure Hash Standard (SHS),” Federal Information Processing
Standards 180. 1993.
[8] NIST, “Secure Hash Standard (SHS),” Federal Information Processing
Standards 180-1, 1995.
[9] B. Preneel, A. Bosselaers and H. Dobbertin, “RIPEMD-160: A
Strengthened Version of RIPEMD,” FSE’96, 1996, LNCS, vol. 1039,
pp. 71–82.
[10] A. Joux, “Multicollisions in Iterated Hash Functions: Application to
Cascaded Constructions,” Crypto’04, 2004, LNCS, vol. 3152, pp. 306-316.
[11] J. Kelsey and B. Schneier, “Second Preimages on n-bit Hash Functions for
Much Less than 2n work,” Eurocrypt’05, 2005, LNCS, vol. 3494,
pp. 474-490.
[12] J. Kelsey and T. Kohno, “Herding Hash Functions and the Nostradamus
Attack,” Eurocrypt’06, 2006, LNCS, vol. 4004, pp. 183–200.
302
JOURNAL OF INFORMATION AND ORGANIZATIONAL SCIENCES
303
TIWARI MERKLE-DAMGÅRD CONSTRUCTION METHOD...
304