CT and MRI Image Fusion Using Non-Subsampled

Contourlet Transform
M. Nazrudeen1, Mrs. Rajalakshmi2
1
Department of CSE, P. A. College of Engineering and Technology, Coimbatore, Tamil Nadu.
2
Assistant Professor, Department of CSE, P. A. College of Engineering and Technology, Coimbatore, Tamil Nadu.
Email: nazru.2013@[Link], rajidha@[Link]

Abstract — Image Fusion is a technique used for various heart, spinal cord and other internal organs. MRI image
application areas. Fusion rule applied for multimodal medical provides better information on soft tissue and normal and
images based on Non-Subsampled Contourlet Transform pathological soft tissues are better visualized. The
(NSCT), it is very useful in the medical application areas. The information provided by Computed Tomography (CT) and
source medical images are first transformed by NSCT, where it Magnetic Resonance Imaging (MRI) is complementary. The
decompose the given source medical image into low frequency
composite image not only provides salient information from
and high frequency components. Two different types of fusion
rules is implemented based on phase congruency method and both images but also reveals the position of soft tissue with
directive contrast technique which is used for fusing the low respect to the bone structure.
frequency coefficients image and high frequency coefficients
image. Finally fused image is obtained by the inverse NSCT A. Different Levels of Fusion
process with all composite coefficients. The proposed fusion
framework provides a better way analysis multimodality
images. The proposed framework is carried out by the three Normally in image fusion three different levels of image
clinical examples of persons affected with Alzheimer, Stroke fusion technique is followed. These include Pixel level
and Recurrent Tumor. With the help of MATLAB source image fusion, Feature level image fusion and Decision level
medical images are decomposed and fusion process is applied.
The inputs are taken from CT and MRI data set and the results
image fusion [14]. This categorization is based according to
are given in the form of MATLAB simulation. merging stage. Feature level image fusion is extracting the
feature from different images that are to be fuse in order to
Index Terms - Multimodal medical image fusion, non- get new image. Decision level image fusion contains
subsampled contourlet transform, phase congruency, directive compact data. It requires the extraction of salient features
contrast. which are depending on their environment such as pixel
I. INTRODUCTION intensities, edges or textures. These similar features from
input images are fused. Decision level image fusion is
In today’s advanced world, medical field have grown a effective for complicated system which is not suitable for
tremendous growth. Many new inventions are implemented general applications. Decision level fusion consists of
in order to diagnosis the patient’s disease in very fast and merging information at a higher level of abstraction it
accurate manner. One such advance technique is Fusion combines the results from multiple algorithms to yield a
process of different medical images. The objective of image final fused decision [3].
fusion is to merge the quality and useful information from In Pixel based image fusion, the fusion process is
different medical images into one fused image which results performed on a pixel-by-pixel basis. It generates a fused
in more accurate and clear information, it combines the image in which information associated with each pixel is
relevant information from a set of images of the same scene determined from a set of pixels in source images to improve
into a single image, resultant fused image will be more the performance of image processing tasks such as
complete and informative than that of input images [1]. A segmentation [15]. Pixel level image fusion is at lowest level
fusion of multimodal images can be very useful for clinical and contains detailed information. Most of the medical
applications such as diagnosis and treatment planning. The image fusion process employs Pixel level image fusion due
input images taken for Fusion process are at different to the advantage of easy implementation, original measured
resolutions and intensity values. This different technique quantity and efficient computation.
helps physicians to extract the features that may not be In this paper, a fusion technique is proposed for
normally visible in a single image by different modalities. multimodal medical images based on Non-Subsampled
There are different types of medical images. Some of Contourlet Transform (NSCT). The main idea is to perform
the examples of medical images are CT, MRI, PET and NSCT on the medical images. The images are decomposed
SPECT images. A CT image is a type of X-ray technology into low frequency coefficients image and high frequency
used for broken bones, blood clots, tumors, blockages and coefficients image. Then fusion rule applied for low
heart disease [2]. CT image provides better information frequency coefficients and high frequency coefficients. The
about denser tissue and structure of tissue bone is better phase congruency method and directive contrast technique
visualized by CT image. MRI image are a type of medical are efficient method for fusion of low frequency coefficients
diagnostic imaging used to look at the blood vessels, brain, image and high frequency coefficients image [13]. The
phase congruency technique provides a contrast and then upsampling the filters accordingly. The advantage of
brightness invariant representation of low frequency this method is that we obtain better low-pass and high-pass
coefficients. The directive contrast technique determines the frequency co-efficients [8].
frequency coefficients from the clear parts of the high
frequency image. The combination of these two methods can 2) Non-Subsampled Directional Filter Bank (NSDFB)
preserve more details in source images and further improve
the quality of fused image. The PSNR and RMSE value of The NSDFB is two-channel non-subsampled filter bank.
proposed technique is better fusion outcome when compared A shift-invariant directional expansion is obtained with a
to other conventional image fusion techniques. Non-Subsampled DFB (NSDFB) [9]. This filter bank is
The rest of the paper is organized as follows. NSCT and constructed by combining the directional fan filter banks and
phase congruency and directive contrast are described in the process is done by eliminating the available
Section II followed by the proposed multimodal medical downsamplers and upsamplers in the DFB. This is done by
image fusion framework in Section III. Experimental results switching off the downsamplers and upsamplers in each two
and discussions are given in Section IV and the conclusion channel filter bank in the DFB tree model and upsample the
and future work are described in Section V. filters respectively.
This Filter bank decompose the images of high
II. PROPOSED SYSTEM frequency images from NSP at each scale and produces
directional sub-images with the same size as the source
This section provides the description of concepts on image [7]. Therefore, the NSDFB offers the NSCT with the
which the proposed framework is based. These concepts multi-direction property and provides us with more precise
include NSCT and Phase Congruency method and Directive directional detail information.
Contrast Technique in NSCT Domain are described as
follows.
Lowpass
Subband
A. Non-Subsampled Contourlet Transform (NSCT) s

The NSCT is a fully multi-scale, multi-direction and

shift invariant expansion of contourlet transform. It is a fast Bandpass
Image directional
implementation process. It is a type of tool in order to subbands
provide a better representation of the contours [4]. NSCT
has an important property of image decomposition. It
achieves similar subband decomposition as similar to that of Bandpass
directional
contourlets, but without downsamplers and upsamplers in it. subbands
Because of its redundancy, the filter design problem of the
NSCT is much less constrained than that of contourlets. This
enables us to design the filters with better frequency Fig. 1. Non-Subsampled Contourlet Transform
selectivity there by achieving a better subband
decomposition [5]. Using the mapping approach we provide B. Phase Congruency
a framework for filter design that ensures good frequency
selectivity in addition to having a fast implementation Phase congruency is an edge detection method. It is
through sequential steps [6]. The NSCT has proven to be particularly robust against changes in illumination and
very efficient in image noise removal and image contrast of image. This approach is based on the Local
enhancement. Energy Model where important features can be found at
NSCT can be divided into two stages for decomposition different points of an image. The Fourier components are
of medical images. First stage is Non-Subsampled Pyramid maximum in phase model. This technique is used for feature
(NSP) and Second, Non-Subsampled Directional Filter Bank detection [10] and blur estimation.
(NSDFB) stage. The algorithm of Phase Congruency is to provide a
good feature localization and noise compensation. Initially,
1) Non- Subsampled Pyramid (NSP) logarithmic Gabor filter banks are applied to the 2D image
and the local amplitude. This method reflects the behaviour
The multiscale property of the NSCT is obtained from a of the image in frequency domain [11]. It has been noted
shift-invariant filtering structure. It achieves subband that edge like features have many of their frequency
decomposition similar to that of the Laplacian pyramid [9]. components in the same phase. It corresponds to the edges in
This is done by using two channel non-subsampled filter an image where sharpness gets changed between light and
banks. The filters for subsequent stages are obtained by dark.
upsampling the filters. This gives the multiscale property Phase congruency was proposed as intensity and
without need of additional filters. contrast invariant dimensionless measure of feature
The decomposition process is done by removing the significance, and used for signal matching and feature
downsamplers and upsamplers in the Laplacian pyramid and extraction.
C. Directive Contrast in NSCT Domain Fusion of Low Frequency Coefficients Image

The directive contrast feature measures the difference of Step 3: In order to fuse low frequency coefficients image,
the intensity value at some pixel from the neighboring the phase congruency method is applied.
pixels. Commonly, the same intensity value looks like a
different intensity value depending on intensity values of The benefit of phase congruency is that it select and
neighboring pixels [12]. Therefore, local contrast is combines the contrast and brightness invariant
developed and is defined as representation contained in the low frequency
coefficients.
L LB LH
C (1) %% Fusion of Low sub band
LB LB % % Phase Congruency
[phaseCongruency_img1, orientation_img1] =
Where L is the local luminance and LB is the luminance phasecong(img1_NSCT_Low);
of the local background. In general, LB is regarded as local figure,imshow(phaseCongruency_img1)
low-frequency, L - LB= LH is treated as local high-frequency. [phaseCongruency_img2, orientation_img2] =
phasecong(img2_NSCT_Low);
This frequency is taken as the pixel value in multiresolution
figure,imshow(phaseCongruency_img2)
domain. img_NSCT_Low_fuse = [];
The problem is that on considering single pixel it is for i = 1:size(phaseCongruency_img1,1)
insufficient to determine the other pixel values. Therefore, for j = 1:size(phaseCongruency_img1,2)
the directive contrast is integrated with the new technique if(phaseCongruency_img1(i,j) >
called sum-modified-Laplacian to get more accurate phaseCongruency_img2(i,j))
important features. img_NSCT_Low_fuse(i,j) = img1_NSCT_Low(i,j);
In general, the larger values of high-frequency else if(phaseCongruency_img1(i,j) <
coefficients correspond to the sharper brightness, edges, phaseCongruency_img2(i,j))
img_NSCT_Low_fuse(i,j) = img2_NSCT_Low(i,j); else
lines and region boundaries. The proper way to select high
img_NSCT_Low_fuse(i,j) =
frequency coefficients is necessary to ensure better (img1_NSCT_Low(i,j)+img2_NSCT_Low(i,j)/2);
information interpretation. Hence, the sum-modified end end
Laplacian is integrated with the directive contrast in NSCT end end
domain to produce accurate salient features. figure, imshow(img_NSCT_Low_fuse,[])

A_low
Source NSCT
Fusion of High Frequency Coefficients Image
Image A Fused Low
A_low Frequency Step 4: In order to fuse high frequency coefficients
Inverse Fused
NSCT Image F image, the directive contrast technique is used.
B_low
Fused High
Source NSCT
Frequency
Image B The most prominent texture and edge information are
B_high selected from high-frequency coefficients and combined
them to form a fused one.
Fig. 2. Block diagram of multimodal medical image fusion
% Fusion of High Sub band
for i= 1 :length(img1_NSCT{1,2})
III. Proposed Multimodal Medical smlA{i} = SML(img1_NSCT_high{i}); end
Image Fusion Framework for i= 1 : length(img2_NSCT{1,2})
smlB{i} = SML(img2_NSCT_high{i}); end
for i=1: length(img1_NSCT{1,2})
IMAGE FUSION USING NSCT ALGORITHM if(img1_NSCT_high{1}~=0)
DirA{i} = smlA{i}/img1_NSCT_high{1}; else
This paper proposes a new image fusion framework for DirA{i} =smlA{i}; end end
multimodal medical images, which is based on the NSCT for i=1: length(img2_NSCT{1,2})
Transformation. The proposed algorithm for Fusion of if(img2_NSCT_high{1}~=0)
medical images (CT and MRI) using NSCT Transformation DirB{i} = smlB{i}/img2_NSCT_high{1}; else
is summarized as follows: DirB{i} =smlB{i};
end end
Step 1: Using NSCT Transformation decompose original
images into proper levels. Step 5: Finally we obtained a Fused Low-frequency image
and Fused High-frequency image.
Step 2: Two different fusion rules are introduced for
combining low frequency coefficients image and Step 6: Inverse Transformation is taken in order to obtain
high frequency coefficients image. Fused Medical Image.
 IV. RESULTS The general requirement of an image fusing process is
to preserve all valid and useful information from the source
images, while at the same time it should not introduce any
distortion in resultant fused image. Performance measures
are used essential to measure the possible benefits of fusion
and also used to compare results obtained with different
algorithms.

A. Peak Signal to Noise Ratio:

PSNR is defined as ratio between the maximum

(a) (b) possible power of a signal and power corresponding to noise
the fidelity of its representation. The PSNR is use to
calculate the similarity between two images. The PSNR
between the reference image R and the fused image F. The
PSNR value is calculated as
2

PSNR 10log ( MAX I ) (2)

10 RMSE

(c) (d) B. Root Mean Square Error (RMSE)

Root Mean Square Error (RMSE) between the fused

image and original image provides error as a percentage of
mean intensity of the original error. The RMSE value is
calculated as:
1 m 1n 1
RMSE [R(i, j) F (i, j)]2 (3)
mn i 0 j 0
(e) (f) Where R(i, j) is the reference image and F(i, j) are the
fused image of CT and MRI respectively, and m and n are
image dimensions. Smaller the value of the RMSE, better
the performance of the fusion algorithm.

TABLE I

COMPARISON OF PSNR AND RMSE VALUES FOR DIFFERENT

MEDICAL IMAGE FUSION TECHNIQUES

(g) Image Set

Different Fusion (CT and MRI scan)
Methods
RMSE PSNR

Medical Image Fusion Based on Ripplet

0.1013 30.45
Transform

Multisensor Image Fusion using Wavelet

0.0756 34.00
Transform

Curvelet Fusion of MRI and CT images 0.0516 36.52

(h)

Fig. 3. Results for Medical Source image: a) CT image. b) Multifocus Image Fusion Based on Non-
MRI image. c) Result for Phase Congruency method. d) 0.0344 38.09
Subsampled Contourlet Transform
Result for Directive Contrast technique. e) Result for fused
low frequency coefficient image. f) Result for fused high Multifocus Image Fusion Based on Non-
frequency coefficient image. g) Result for proposed Fused Subsampled Contourlet Transform and 0.0211 41.66
Medical image. h) Result for Color representation of Directive Contrast Method
proposed Fused Medical image.
 FUTURE WORK
0.12 The simulation results show the superiority of the NSCT
0.1 and future going to compare the result with the DTCWT
0.08 transform. The higher the value of the PSNR and RMSE is
the better performance of the fusion algorithm.
0.06
0.04
0.02 RMSE REFERENCES
0
[1] L. Yang, B. L. Guo, and W. Ni, “Multimodality medical image
fusion based on multiscale geometric analysis of contourlet
transform,” Neurocomputing, vol. 72, pp. 203–211, 2008.

[2] T. Li and [Link], “Biological image fusion using a NSCT

based variable-weight method,” Inf. Fusion, vol. 12, no. 2, pp.
85–92, 2011.

[3] F. E. Ali, I. M. El-Dokany, A. A. Saad, and F. E. Abd El-Samie,

Fig. 4. Comparison of RMSE of Various Image Fusion “Curvelet fusion of MR and CT images,” Progr. Electromagn.
Res. C, vol. 3, pp. 215–224, 2008.
Techniques.
[4] X. Qu, J. Yan, H. Xiao, and Z. Zhu, “Image fusion algorithm
based on spatial frequency-motivated pulse coupled neural
50 networks in nonsubsampled contourlet transform domain,” Acta
40 Automatica Sinica, vol. 34, no. 12, pp. 1508–1514, 2008.

30 [5] Q. Zhang and B. L. Guo, “Multifocus image fusion using the

20 nonsubsampled contourlet transform,” Signal Process., vol. 89,
no. 7, pp. 1334–1346, 2009.
10
PSNR [6] Y. Chai, H. Li, and X. Zhang, “Multifocus image fusion based
0
on features contrast of multiscale products in nonsubsampled
contourlet transform domain,” Optik, vol. 123, pp. 569–581,
2012.

[7] S. Yang, M. Wang, Y. Lu, W. Qi, and L. Jiao, “Fusion of

multiparametric SAR images based on SW-nonsubsampled
contourlet and PCNN,” Signal Process., vol. 89, no. 12, pp.
2596–2608, 2009.

Fig. 4. Comparison of PSNR of Various Image Fusion [8] Y. Chai, H. Li, and X. Zhang, “Multifocus image fusion based
Techniques. on features contrast of multiscale products in nonsubsampled
contourlet transform domain,” Optik—Int. J. Light Electron Opt.,
vol. 123, no. 7, pp. 569–581, 2012.
V. CONCLUSION
[9] A. L. da Cunha, J. Zhou, and M. N. Do, “The nonsubsampled
In this paper, an image fusion technique is proposed for contourlet transform: Theory, design, and applications,” IEEE
multi-modal medical images of CT and MRI, which is based [Link] Process., vol. 15, no. 10, pp. 3089–3101, Oct. 2006.
on Non-Subsampled Contourlet Transform, Phase
[10] P. Kovesi, “Image features from phase congruency,” Videre: J.
congruency model and directive contrast technique. First Comput. Vision Res., vol. 1, no. 3, pp. 2–26, 1999.
medical images are decomposed into low frequency
coefficients image and high frequency coefficients image [11] P. Kovesi, “Phase congruency: A low-level image invariant,”
using NSCT transformation. For fusion process two different Psychol. Res. Psychologische Forschung, vol. 64, no. 2, pp.
136–148, 2000.
rules are applied. First low frequency coefficient images are
fused using phase congruency technique (low band fusion [12] G. Bhatnagar and B. Raman, “A new image fusion technique
rule). Next the high coefficient images are fused using based on directive contrast,” Electron. Lett. Comput. Vision
directive contrast technique (high band fusion rule). Finally Image Anal., vol. 8, no. 2, pp. 18–38, 2009.
the fused coefficients are reconstructed by performing the [13] [Link], Z. Dali, and Y. Pingfan, “Medical image fusion by
inverse process of DWT. By this more information can be wavelet transform modulus maxima,” Opt. Express, vol. 9, pp.
preserved in the fused image with improved quality. The 184–190, 2001.
proposed method is advantage over the other classical fusion
[14] V. Barra and J. Y. Boire, “A general framework for the fusion of
modal such as IHS and PCA, Brovey and Multiscale anatomical and functional medical images,” Neuro Image, vol.
transform methods such as DWT and SF (spatial frequency), 13, no. 3, pp. 410–424, 2001.
Stationary Wavelet transform. The proposed algorithm is
evaluated with the qualitative analytical measurement of [15] S. Das, M. Chowdhury, and M. K. Kundu, “Medical image
fusion based on ripplet transform type-I,” Progr. Electromagn.
PSNR and RMSE Values. Res. B, vol. 30, pp. 355–370, 2011.