Deep Joint Source Channel Coding for Wireless

Image Transmission with OFDM

Mingyu Yang, Chenghong Bian, and Hun-Seok Kim
University of Michigan, Ann Arbor, USA

Abstract—We present a deep learning based joint source

channel coding (JSCC) scheme for wireless image transmission
arXiv:2101.03909v2 [[Link]] 24 May 2021

over multipath fading channels with non-linear signal clipping.

The proposed encoder and decoder use convolutional neural
networks (CNN) and directly map the source images to complex-
valued baseband samples for orthogonal frequency division
multiplexing (OFDM) transmission. The proposed model-driven
machine learning approach eliminates the need for separate
source and channel coding while integrating an OFDM datapath
to cope with multipath fading channels. The end-to-end JSCC
communication system combines trainable CNN layers with non-
trainable but differentiable layers representing the multipath
channel model and OFDM signal processing blocks. Our results
show that injecting domain expert knowledge by incorporating
OFDM baseband processing blocks into the machine learn-
ing framework significantly enhances the overall performance
compared to an unstructured CNN. Our method outperforms Fig. 1. Block diagrams of wireless image transmission schemes. Top: Tra-
conventional schemes that employ state-of-the-art but separate ditional separate source and channel coding scheme. Bottom: Deep learning
source and channel coding such as BPG and LDPC with OFDM. based joint source channel coding (JSCC) scheme.
Moreover, our method is shown to be robust against non-linear
signal clipping in OFDM for various channel conditions that do
not match the model parameter used during the training. general. Considering the complexity and power constraints
Index Terms—Joint source channel coding, deep neural net-
works, OFDM, model-driven machine learning
of practical systems, sub-optimal separate source and channel
coding is regarded as a standard scheme.
I. I NTRODUCTION In recent years, deep learning has been successfully applied
to a wide range of areas such as computer vision [2] [3]
Shannon’s separation theorem [1] states that, as the size and natural language processing [4] [5], and it achieved
of the transmitted message goes to infinity for memory-less significant performance improvement over analytical (non-
channels, it is optimal to split the communication task into data-driven) algorithms. Its ability to extract complex features
(i) removing the redundant information of the source as much from images has led to applications of deep learning to joint
as possible and (ii) re-introducing redundancy for message source channel coding (JSCC) over simple channel models
reconstruction in the presence of channel noise. Based on this such as additive white Gaussian noise (AWGN) [6] [7] [8]
famous theorem, most modern systems for wireless image and binary symmetric channel (BSC) [9] [10]. These methods
transmission first compress the image with a source coding demonstrate the feasibility of practical machine learning based
algorithm (e.g., JPEG, WebP, BPG) and then encode the bit JSCC schemes that yield better performance than prior sep-
stream with a source-independent channel code (e.g., LDPC, arate coding schemes. However, there is no JSCC work that
Polar, Turbo, etc.) as shown in Figure 1. This separate source extends to realistic and challenging multipath fading channels.
and channel coding is attractive for practical communication
In this paper, we propose a deep learning based JSCC
systems because of the convenient modularity. However, in
scheme for wireless image transmission under multipath fad-
practice, we often transmit a finite number of bits and that
ing channels. Inspired by prior works [6] [7] [8], we employ
breaks the assumption of Shannon’s separation theorem. For
a pair of encoder and decoder that directly encodes images
a finite length image source, reconstruction quality becomes
as complex valued channel input samples and recovers them
a joint function of the source coding distortion and channel
from noisy channel outputs. Moreover, adopting the idea of
coding error. Thus it is difficult to design an optimal scheme
model-driven deep learning approach [11], we extend the
that balances these two sources of the quality degradation.
JSCC framework to combine trainable CNN layers as well as
Besides, the optimal maximum-likelihood detection for the
non-trainable but differentiable layers representing a multipath
joint source and channel coded information is NP-hard in
channel model and OFDM baseband processing blocks. As
This work was funded in part by DARPA YFA #D18AP00076 and NSF the authors of other prior work reported in [12] [13], incor-
CAREER #1942806. porating the domain expert knowledge improves the training
Fig. 2. Left: Flow diagram for the proposed JSCC framework with OFDM extension. Right: Two decoder structures considered in this paper. IMPLICIT:
directly concatenate everything and feed them to the decoder network. EXPLICIT: introduce explicit channel estimation and equalization methods with
additional residual connections.

speed and enhances the performance over a conventional this issue, model-driven deep learning frameworks combine
auto-encoder approach without an explicit domain-knowledge- standard neural networks with analytical/mathematical models
driven structure. We show that as we impose additional and signal processing blocks guided by the expert knowledge
domain-knowledge-driven structure such as channel estimation to shorten the training time and also to improve the per-
and equalization into the decoder, its performance further formance. Combining machine learning with expert domain
improves. In the proposed scheme, OFDM processing blocks knowledge has shown great success in many applications such
(FFT, cyclic prefix insertion, etc.) and multpath channel model as radio transformer network (RTN) [12], symbol modulation
are both implemented as differentiable layers so that the end- and demodulation for OFDM [13], channel estimation [19],
to-end framework can be trained with multipath fading as well and peak-to-average power Ratio (PAPR) reduction [20].
as non-linear signal clipping at the transmitter. Our results
show that the proposed JSCC method outperforms the baseline III. P ROPOSED M ETHOD
separate source and channel scheme with a state-of-the-art A. Deep JSCC in multipath fading channels
image compression method and error correction code applied The general framework for deep learning based JSCC is
to OFDM. Furthermore, we show that our proposed method shown in Figure 1. In our approach, a pair of encoder Eθ and
is robust for a wide range of SNR and channel conditions that decoder Dφ directly map the source input x to complex-valued
the neural network has not seen during the training process. (modulated) baseband samples y and estimate the source signal
II. R ELATED W ORK x̂ from noisy multipath fading channel outputs ŷ. Here, θ
A. Deep Joint Source Channel Coding and φ represent the parameters for the encoder and decoder
neural networks respectively. The channel can be represented
Deep JSCC was first studied in [6] where the authors with the conditional probability p(ŷ|y; ǫ) where ǫ denotes the
incorporate both source and channel coding into an auto- parameters that describe the channel statistics. The channel is
encoder structure, and jointly learn source compression and defined in the continuous time domain assuming interpolation
error correction coding through the back-propagation opti- and Nyquist sampling for the channel inputs and outputs.
mization. The proposed deep JSCC algorithm outperforms We parameterize the multipath fading channel using a
conventional separate schemes under AWGN and Rayleigh flat channel transfer function;
fading channel. It can be further extended with feedback [8]
or progressive transmission [7]. Similarly, deep JSCC on BSC ŷ = h(y; σ02 , . . . , σL−1
2
, σ2 ) = h ∗ y + w (1)
and binary erasure channels have been studied in [9] [10]. L
Due to their discrete nature, the authors perform the end-to- where ∗ denotes the convolution operation, h ∈ C denotes
end optimization in a variational manner and optimize a lower the sample space channel impulse response, and L is the num-
bound of mutual information between the source images and ber of multipaths. w ∼ CN (0, σ 2 Ik ) represents the additive
noisy channel outputs. In additional to images, JSCC has been Gaussian noise. Each path experiences independent Rayleigh
studied for Gaussian source [14] and text source [15]. fading satisfying hl ∼ CN (0, σl2 ) for l = 0, 1, . . . , L − 1.
The power for each path follows an exponential decay profile
l
B. Model-driven Machine Learning for Communications σ 2 = α e− γ where αl is a normalization coefficient to satisfy
Pl L−1 l 2
Recent years, deep learning methods have been widely l=0 σl = 1. γ is the time decay (or delay spread) constant.
applied to wireless communications [16] [17] [18]. However, Note that the channel transfer function is fully differentiable
as discussed in [11], these data-driven methods simply treat given realizations of random channel parameters. It means
standard neural networks as a black box, which require a that the gradients from the decoder can propagate back to
large amount of data and long training time. To address the encoder through a particular realization of the multipath
fading channel when the end-to-end system is trained with Ŷp and Ŷ . The loss function of this learning problem can then
gradient descent methods. For the gradient descent training of be expressed as:
the encoder and decoder pair, we use random realizations of
L(θ, φ) = Ep(x,Ŷ ,Ŷp ) [d(Dφ (Eθ (x), Ŷ , Ŷp ), x)] (5)
the channel model described in this section.
where Ep [ ] is the expected value over distribution p,
B. JSCC with OFDM Extension and Signal Clipping and d denotes the distortion calculated by x and x̂(=
We extend the JSCC framework to an OFDM-based sys- Dφ (Eθ (x), Ŷ , Ŷp )). The goal is to find θ∗ and φ∗ that mini-
tem to efficiently mitigate the multipath fading channel with mize the loss function L(θ, φ).
simple single-tap frequency domain equalization. We assume It must be noted that including OFDM signal process-
that each source image x is transmitted in a single OFDM ing into the proposed JSSC framework is possible because
packet that contains Np pilot symbols and Ns information IDFT/DFT and CP removal/insertion can be treated as linear
symbols. For channel estimation, we adopt block-type pilots layers in the neural network with fixed parameters. During
by sending known symbols on all subcarriers. For simplicity, back-propagation, gradients can pass through these (I)DFT/CP
we assume that the transmitter and receiver are synchronized in linear layers. Similarly, the proposed JSSC framework learns
time (within the cyclic-prefix length Lcp ) without any carrier the encoder and decoder networks in the presence of signal
frequency offset. clipping at the transmitter which is similar to a clipped ReLu
The OFDM extension to the JSCC framework is shown activation function.
in Figure 2. With OFDM, the encoder output becomes the
C. Decoder Design with Domain Knowledge
frequency domain symbols Y ∈ CNs ×Lf f t fed into the OFDM
transmitter. The pilot symbols Yp ∈ CNp ×Lf f t are known To estimate the source information, one possible approach is
to both the transmitter and receiver. Here, Lf f t denotes the to concatenate Ŷ , Yp and Ŷp to directly feed them into a deep
number of subcarriers in an OFDM symbol. We apply inverse (‘upscale’) neural network. This is illustrated in Figure 2 as
discrete Fourier transform (IDFT) to each OFDM symbol and the IMPLICIT method. This approach relies on the deep neural
append the cyclic-prefix (CP). network to learn the underlying relationships in (3) and (4) to
One notable drawback of OFDM is the high PAPR that perform the (sub)optimal estimation of the source. We name
causes excessive power consumption at the power amplifier this method IMPLICIT because it implicitly learns traditional
[21]. A number of solutions have been proposed to reduce the channel estimation and equalization processes in the decoder
PAPR. Among them, signal clipping is one of the simplest network. However, this method treats the entire decoder as a
and extensively studied methods [22] [23]. Suppose we have blockbox, and hence it faces the problems of slow convergence
the original TX sample ypreclip [n] = A[n]ejφ[n] where A[n] and sub-optimal performance as argued in [11]. As an alterna-
is the amplitude and φ[n] is the phase. The clipping operation tive solution, we propose a decoder structure shown in Figure
can be expressed as: 2 bottom right where explicit signal processing kernels such
as channel estimation and equalization are included as pre-
( √
A[n]ejφ[n] when A[n] ≤ ρ Ps processing steps in front of the neural network based decoder.
y[n] = √ (2) Due to the signal clipping and other non-linear activations in
ρ Ps ejφ[n] otherwise
the transmitter, the manually-designed channel estimation and
where ρ denotes the clipping ratio (CR) and Ps is the input equalization methods may not be optimal. Thus, we further
signal power. In the proposed JSCC, this clipping operation introduce two residual connections with light-weight neural
is considered as an additional non-linear activation function networks (‘subnets’) so that they can learn and compensate
as shown in Fig. 3 and the gradients propagate through this residual errors in the channel estimation and equalization. We
clipping layer during the JSCC encoder-decoder pair training. name this method EXPLICIT. For simplicity, we use the per-
After clipping, the transmit signal y ∈ C(Ns +Np )(Lf f t +Lcp ) channel MMSE channel estimation method [13] which does
propagates through the multipath channel described in (1). not need second order statistics of the channel and matrix
Once the receiver obtains the noisy channel output ŷ, it inversion. For equalization, we adopt a conventional MMSE
removes the CP, and applies DFT to produce the frequency equalizer.
domain pilots Ŷp and data symbols Ŷ . If there was no clipping
IV. T RAINING AND E VALUATION
(either deliberate or amplifier-inherent), the following holds for
pilot and information symbols: We test our proposed method using CIFAR-10 and CelebA
datasets. The CIFAR-10 dataset contains 60,000 32×32-pixel
Ŷp [i, k] = H[k]Yp [i, k] + W [i, k] (3) images whereas CelebA contains more than 200,000 celebrity
images. We scaled and cropped CelebA images to 64×64-
Ŷ [j, k] = H[k]Y [j, k] + V [j, k] (4) pixels. For testing, we take 10,000 images from each test
dataset (unused for training) and transmit each image 5 times
where H denotes the channel frequency response for the kth through different random realizations of the multipath channel.
subcarrier, and both W and V denote noise samples. The We use both PSNR and SSIM to evaluate the reconstruction
decoder is trained to estimate transmitted source x̂ given Yp , quality. We test our JSCC method on CIFAR-10 using the
 Learning curves
7
Direct
IMPLICIT
6 EXPLICIT

Loss
3

0
0 2000 4000 6000 8000 10000 12000
Iterations

Fig. 4. Learning curves for three methods with a SNR of 20dB and Ns = 6
for CelebA dataset.
Fig. 3. Network structure for the proposed method. There is a batch
normalization layer between each convolutional layer and ReLU activation
function (although not shown in the figure).
A. Effect of applying domain knowledge
TABLE I To quantify the effect of incorporating domain knowledge
CIFAR-10 AND C ELEBA EVALUATION (T OP : PSNR, B OTTOM : SSIM) into our framework, we compare our method with a neural
network-only scheme which we call ‘direct’ where symbols
Dataset CIFAR-10, CPP=0.21 CelebA, CPP=0.05 are transmitted through the multipath channel without OFDM
SNR 0dB 10dB 20dB 0dB 10dB 20dB layers. All compared methods are tested with both CIFAR10
Direct w/o OFDM 21.05 24.93 26.21 21.86 24.76 25.73
IMPLICIT 21.82 26.7 30.54 22.27 26.54 29.37
and CelebA under different SNRs and the coding rate is
EXPLICIT 22.31 27.91 31.69 22.92 27.27 30.31 kept the same for each dataset (CPP is 0.21 and 0.05 for
CIFAR-10 and CelebA, respectively). The results are shown
Dataset CIFAR-10, CPP=0.21 CelebA, CPP=0.05 in Table I. Including OFDM (I)DFT/CP processing layers, our
SNR 0dB 10dB 20dB 0dB 10dB 20dB IMPLICIT and EXPLICIT method attain superior PSNR and
Direct w/o OFDM 0.697 0.84 0.871 0.683 0.788 0.811 SSIM than the direct method without OFDM layers especially
IMPLICIT 0.709 0.893 0.952 0.702 0.85 0.908
EXPLICIT 0.727 0.915 0.962 0.73 0.867 0.924 for high SNRs. As we introduce more domain knowledge into
EXPLICIT configuration, the system experiences additional
performance gain. Note that the subnets in EXPLICIT (Fig.
2) only introduce 0.2% additional parameters than IMPLICIT
network structure shown in Figure 3, which follows the design method to obtain the gain shown in Table I.
principles in [24]. Because of the image size difference, As shown in Figure 4, significant training efficiency gain
networks for CelebA have one more down-sampling module at is obversed for IMPLICIT and EXPLICIT compared to the
the encoder and one more upsampling module at the decoder. ‘direct’ method. With OFDM (I)DFT/CP layers, IMPLICIT
The subnets in EXPLICIT method follow a Conv-BatchNorm- and EXPLICIT attain significantly (2.5×) lower loss function
ReLu-Conv-BatchNorm structure just like the residual block in values with faster (5×) training convergence time for CelebA
the main network shown in Figure 3. dataset. The training time is similar for IMPLICIT and EX-
We adopt mean-squared-error (MSE) as our distortion mea- PLICIT schemes but EXPLICIT method converges to a 20%
sure (5) for the network training. For both datasets, we use a lower loss function value for CelebA training.
batch size of 128 and train the neural networks using ADAM
with β1 = 0.5 and β2 = 0.999. For CIFAR-10, we apply B. Comparison with a separate coding scheme
a learning rate of 10−3 and train for 400 epochs with linear In this section, we compare our proposed JSCC method
learning rate decaying for the last 200 epochs. For CelebA, with a baseline separate source and channel coding scheme.
the learning rate is set to 5 × 10−4 and we train the end-to- The baseline separate coding scheme uses the state-of-the-art
end system for 60 epochs with linear learning rate decaying BPG [25] as the image codec and LDPC in the IEEE 802.11n
for the last 30 epochs. WiFi standard [26] as the channel code. We test with three
The OFDM and channel parameters are set to Lf f t = different LDPC codes (972, 1944), (1296, 1944), (1458, 1944)
64, Lcp = 16, Np = 2, L = 8, and γ = 4. Note that the which correspond to rates 1/2, 2/3 and 3/4 respectively.
coding rate of our method depends on the number of samples We use BPSK, QPSK, 16QAM, and 64QAM for OFDM
after encoding. For a source image of the size H × W × C modulation applied to BPG and LDPC coded bit sequences.
pixels (C is the number of color channels), the coding rate is Then, we enumerated all possible modulation and coding rate
(Np +Ns )(Lf f t +Lcp )
HW C (channel-use per pixel, CPP). The length combinations to identify the optimal configuration for the
of CP has to be larger than the delay spread of the channel to baseline separate coding scheme. The channel estimation for
avoid inter-symbol interference. this baseline is based on per-channel MMSE, which is also
 SNR = 5dB, PSNR SNR = 5dB, SSIM SNR = 15dB, PSNR SNR = 15dB, SSIM
1 1
28 34
0.9 0.95
26
32
0.8
24 0.9
PSNR(dB)

PSNR(dB)
30

SSIM

SSIM
22 0.7
0.85
28
20 0.6 IMPLICIT
IMPLICIT IMPLICIT IMPLICIT
EXPLICIT 0.8 EXPLICIT
EXPLICIT EXPLICIT 26
18 0.5 BPG+2/3LDPC+16QAM (perfect CSI) BPG+2/3LDPC+16QAM (perfect CSI)
BPG+1/2LDPC+BPSK (perfect CSI) BPG+1/2LDPC+BPSK (perfect CSI)
BPG+1/2LDPC+BPSK (MMSE) BPG+2/3LDPC+16QAM (MMSE) BPG+2/3LDPC+16QAM (MMSE)
BPG+1/2LDPC+BPSK (MMSE) 24 0.75
16 BPG+1/2LDPC+QPSK (perfect CSI) BPG+1/2LDPC+64QAM (perfect CSI) BPG+1/2LDPC+64QAM (perfect CSI)
0.4 BPG+1/2LDPC+QPSK (perfect CSI)
BPG+1/2LDPC+QPSK (MMSE) BPG+1/2LDPC+64QAM (MMSE) BPG+1/2LDPC+64QAM (MMSE)
BPG+1/2LDPC+QPSK (MMSE)
14 22 0.7
4 5 6 7 8 9 10 11 12 4 5 6 7 8 9 10 11 12 4 5 6 7 8 9 10 11 12 4 5 6 7 8 9 10 11 12
Number of information symbols (N ) Number of information symbols (N ) Number of information symbols (N ) Number of information symbols (N )
s s s s

Fig. 5. Performance of the proposed method on CIFAR-10 with respect to Fig. 6. Performance of the proposed method on CIFAR-10 with respect to
the number of information symbols (Ns ) for a fixed SNR of 5dB the number of information symbols (Ns ) for a fixed SNR of 15dB

Ns = 6, PSNR Ns = 6, SSIM
34 1

used in our EXPLICIT scheme. The estimated channel is then 32

30 0.9

used to calculate the log likelihood ratio (LLR) for LDPC 28

PSNR(dB)
0.8
decoding in the baseline separate coding scheme. 26

SSIM
24

We compare PSNR and SSIM of the proposed JSSC and 22

IMPLICIT
0.7
IMPLICIT
20 EXPLICIT EXPLICIT
the separate coding baseline at the same/similar rate and SNR. 18
BPG+LDPC+QAM (perfect CSI)
BPG+LDPC+QAM (MMSE)
0.6 BPG+LDPC+QAM (perfect CSI)
BPG+LDPC+QAM (MMSE)

Because BPG cannot achieve arbitrary compression rates, we 16

0.5
0 5 10 15 20 0 5 10 15 20
exhaustively search for the BPG parameter to obtain a rate SNR (dB) SNR (dB)

closest to the target. For fair comparison, we did not include

Fig. 7. Performance of the proposed method on CIFAR-10 with respect to
the BPG headers in the BPG rate computation. When BPG- SNR for a fixed number of information symbols (Ns = 6)
LDPC encoded packets fail to reconstruct images due to
fatal bit errors, we simply assume they achieve the average
PSNR/SSIM of decodable images, giving an advantage to the reasonable PSNR/SSIM at the given rate. It is observed that the
baseline separate coding scheme. proposed EXPLICIT JSCC method consistently outperforms
Figure 5 and 6 compare the performance of deep JSCC the baseline with >3dB SNR gain for the same PSNR. The
schemes and baseline approaches with respect to the number SNR gain is even higher when the image quality is measured
of information symbols Ns in different SNR regimes using in SSIM.
CIFAR-10 dataset. Smaller Ns indicates more aggressive
(lower) rate to send an entire image with a fewer number of C. Robustness Analysis
data symbols. For comparison, we also show the performance Finally, we evaluate robustness of the JSCC with respect to
of baseline separate coding schemes with perfect channel state the signal clipping. We also evaluate the impact of mismatched
information (CSI) for error-free channel estimation. Note that, channel statistics such as SNR or the number of multipaths
for the ≤5dB SNR case, BPG with 1/2 LDPC+BPSK cannot used during the JSCC training vs. actual testing.
achieve the rate of Ns = 4 for the full image. Figure 5 Figure 8 shows the effect of clipping to the performance
and 6 show our deep JSSC consistently outperforms baseline of the EXPLICIT JSCC and baseline method with various
separate coding schemes especially for the low SNR and low clipping ratios (CR) of ρ = 1, 1.4, and ∞ as defined in
rate regime. Note that the performance of the baseline method (2). Note that for the baseline, we adopt the methods in [22]
approaches to that of the deep JSSC for some high rates when [23] to calculate the LLR including the non-linear distortion
perfect CSI is given for their advantage. For the deep JSSC introduced by clipping. It is observed that our JSCC provides a
(both IMPLICIT and EXPLICIT), we never provide perfect graceful degradation in PSNR/SSIM than the baseline method
CSI to the decoder. With imperfect CSI (same as in JSSC), as more aggressive clipping is applied. And same as the
the perfomrnace of the baseline scheme is significantly worse previous section, our method outperforms the baseline in all
(e.g., >3dB PSNR) at the same low rate target. It is known range of SNRs in the presence of clipping.
that SSIM is a better metric than PSNR to quantify human Figure 9 left shows the JSCC performance when it is trained
perception of an image. Our deep JSSC provides a significant for a specific SNR (5dB or 15dB) and tested with a wide
SSIM gain compared to the baseline although it is trained to range of SNR from 0dB to 20dB. It is observed that the
minimizes the MSE loss function, not SSIM. model trained with 5dB performs better for the low SNR
For the evaluation in Figure 7, we fix the rate to Ns = 6 regime while the model trained with 15dB performs better
for the the CIFAR-10 dataset (0.21 channel-use per pixel) in the high SNR region. More balanced JSCC performance is
and evaluate the reconstructed image quality for a wider SNR obtained when the model is trained with random SNR values
range. For the baseline method, we choose at each SNR point covering a wide range. The EXPLICIT JSCC model in Figure
the optimal combination of the OFDM modulation QAM size 9 right was trained with an L = 8 multipath channel model and
and LDPC rate that yields the best performance. For 0dB SNR, tested with a wide range of L but the same noise level. Two
the baseline method without perfect CSI failed to provide curves show the performance at 5dB and 15dB SNR cases.
 Ns = 6, PSNR Ns = 6, SSIM
32 1 [4] Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bah-
30
danau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio, “Learning
28
0.9
phrase representations using rnn encoder-decoder for statistical machine
translation,” arXiv preprint arXiv:1406.1078, 2014.
PSNR(dB)

0.8
26
[5] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion

SSIM
24
0.7
EXPLICIT (no clipping)
Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin, “Attention
22
EXPLICIT (no clipping)
EXPLICIT (CR=1.4)
EXPLICIT (CR=1.4)
EXPLICIT (CR=1)
is all you need,” in Advances in neural information processing systems,
EXPLICIT (CR=1)

20
BPG+LDPC+QAM (no clipping)
0.6 BPG+LDPC+QAM (no clipping)
BPG+LDPC+QAM (CR=1.4) 2017, pp. 5998–6008.
BPG+LDPC+QAM (CR=1.4) BPG+LDPC+QAM (CR=1)
BPG+LDPC+QAM (CR=1) [6] Eirina Bourtsoulatze, David Burth Kurka, and Deniz Gündüz, “Deep
18 0.5
0 5 10 15 20 0 5 10 15 20 joint source-channel coding for wireless image transmission,” IEEE
SNR (dB) SNR (dB)
Transactions on Cognitive Communications and Networking, vol. 5, no.
3, pp. 567–579, 2019.
Fig. 8. Performance of the proposed method on CIFAR-10 with respect to [7] David Burth Kurka and Deniz Gündüz, “Successive refinement of
SNR for different clipping ratios. Ns = 6. images with deep joint source-channel coding,” in SPAWC. IEEE, 2019,
pp. 1–5.
1 1
[8] David Burth Kurka and Deniz Gündüz, “Deepjscc-f: Deep joint source-
0.95 EXPLICIT (Trained with L=8, Test at 5dB SNR) channel coding of images with feedback,” IEEE Journal on Selected
EXPLICIT (Trained with L=8, Test at 15dB SNR)
0.9
0.95
Areas in Information Theory, 2020.
0.85 [9] Kristy Choi, Kedar Tatwawadi, Aditya Grover, Tsachy Weissman, and
Stefano Ermon, “Neural joint source-channel coding,” in ICML. PMLR,
SSIM

SSIM

0.8
0.9
0.75 2019, pp. 1182–1192.
0.7 [10] Yuxuan Song, Minkai Xu, Lantao Yu, Hao Zhou, Shuo Shao, and Yong
EXPLICIT (Trained with 5dB SNR)
0.85
0.65 EXPLICIT (Trained with 15dB SNR)
EXPLICIT (Trained with random SNR)
Yu, “Infomax neural joint source-channel coding via adversarial bit flip,”
0.6 BPG+LDPC+QAM arXiv preprint arXiv:2004.01454, 2020.
0.55
0 5 10 15 20
0.8
2 4 6 8 10 12 14 16 18 [11] Hengtao He, Shi Jin, Chao-Kai Wen, Feifei Gao, Geoffrey Ye Li,
SNR (dB) Number of paths (L) and Zongben Xu, “Model-driven deep learning for physical layer
communications,” IEEE Wireless Communications, vol. 26, no. 5, pp.
Fig. 9. Robustness test for the JSCC with CIFAR-10 images and Ns = 6. 77–83, 2019.
Left: training vs. testing SNR mismatch scenarios, Right: multipath channel [12] Timothy O’Shea and Jakob Hoydis, “An introduction to deep learning
model parameter (L) mismatch scenarios with SNR = 5dB or 15dB. for the physical layer,” IEEE Transactions on Cognitive Communications
and Networking, vol. 3, no. 4, pp. 563–575, 2017.
[13] Alexander Felix, Sebastian Cammerer, Sebastian Dörner, Jakob Hoydis,
It is confirmed that our JSCC method is relatively insensitive and Stephan Ten Brink, “Ofdm-autoencoder for end-to-end learning of
communications systems,” in 2018 IEEE 19th International Workshop
to the mismatch in the number of multipaths of the channel on Signal Processing Advances in Wireless Communications (SPAWC).
model for the training vs. testing. IEEE, 2018, pp. 1–5.
[14] Yashas Malur Saidutta, Afshin Abdi, and Faramarz Fekri, “Joint
V. C ONCLUSION source-channel coding for gaussian sources over awgn channels using
variational autoencoders,” in ISIT. IEEE, 2019, pp. 1327–1331.
In this paper, we present a deep learning based JSCC [15] Nariman Farsad, Milind Rao, and Andrea Goldsmith, “Deep learning
scheme for wireless image transmission over multipath fading for joint source-channel coding of text,” in ICASSP. IEEE, 2018, pp.
channels. We extend the JSCC scheme with explicit OFDM 2326–2330.
[16] Tianqi Wang, Chao-Kai Wen, Hanqing Wang, Feifei Gao, Tao Jiang,
layers and signal processing steps driven by the expert domain and Shi Jin, “Deep learning for wireless physical layer: Opportunities
knowledge. With addition explicit steps, the JSCC train- and challenges,” China Communications, vol. 14, no. 11, pp. 92–111,
ing convergence time improves with enhanced performance 2017.
[17] Zhijin Qin, Hao Ye, Geoffrey Ye Li, and Biing-Hwang Fred Juang,
compared to naive IMPLICIT approaches without exploiting “Deep learning in physical layer communications,” IEEE Wireless
domain knowledge. Through extensive experimental simula- Communications, vol. 26, no. 2, pp. 93–99, 2019.
tions, we show that the deep JSCC method outperforms the [18] Guan Gui, Hongji Huang, Yiwei Song, and Hikmet Sari, “Deep
learning for an effective nonorthogonal multiple access scheme,” IEEE
conventional separate source and channel coding schemes with Transactions on Vehicular Technology, vol. 67, no. 9, pp. 8440–8450,
OFDM especially for low SNR and low rate regimes. The pro- 2018.
posed JSCC framework effectively learns source compression [19] Xuanxuan Gao, Shi Jin, Chao-Kai Wen, and Geoffrey Ye Li, “Comnet:
Combination of deep learning and expert knowledge in ofdm receivers,”
as well as protection against noise in challenging multipath IEEE Communications Letters, vol. 22, no. 12, pp. 2627–2630, 2018.
channels. Moreover, the JSCC framework incorporates delib- [20] Minhoe Kim, Woongsup Lee, and Dong-Ho Cho, “A novel papr
erate signal clipping during the training process to significantly reduction scheme for ofdm system based on deep learning,” IEEE
Communications Letters, vol. 22, no. 3, pp. 510–513, 2017.
reduce the PAPR with graceful performance degradation. Our [21] Xiaodong Li and Leonard J Cimini, “Effects of clipping and filtering
approach is shown to be robust when the evaluation channel on the performance of ofdm,” in 1997 IEEE 47th Vehicular Technology
model characteristics are mismatched to those used during the Conference. Technology in Motion. IEEE, 1997, vol. 3, pp. 1634–1638.
[22] Hideki Ochiai and Hideki Imai, “Performance analysis of deliberately
training. clipped ofdm signals,” IEEE Transactions on communications, vol. 50,
no. 1, pp. 89–101, 2002.
R EFERENCES [23] Hun Seok Kim and Babak Daneshrad, “Power optimized pa clipping for
[1] Claude E Shannon, “A mathematical theory of communication,” The mimo-ofdm systems,” IEEE transactions on wireless communications,
Bell system technical journal, vol. 27, no. 3, pp. 379–423, 1948. vol. 10, no. 9, pp. 2823–2828, 2011.
[2] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton, “Imagenet [24] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros, “Image-
classification with deep convolutional neural networks,” in Advances in to-image translation with conditional adversarial networks,” in Proceed-
neural information processing systems, 2012, pp. 1097–1105. ings of the IEEE conference on computer vision and pattern recognition,
[3] Karen Simonyan and Andrew Zisserman, “Very deep convolu- 2017, pp. 1125–1134.
tional networks for large-scale image recognition,” arXiv preprint [25] Fabrice Bellard, “BPG image format,” [Link]
arXiv:1409.1556, 2014. [26] tavildar, “Ldpc codes,” [Link]