[CHAPTER-4] [Digital Communication] Information theory

Introduction

Till now we have studied communication primarily in terms of "signal". How signal is
affected by noise, interference and so on. However, signal theory does not give a
concrete view of the fundamental communication process of information transfer. To
represent the digital communication system in terms of information, Claude Shannon
in 1948 came up with a theory known as ‘Mathematical Theory of Communication’ in
which he concentrated on the information content of the message rather than on the
signal. His approach was soon renamed as Information theory, and deals with
mathematical modeling and analysis of a communication system rather than with
physical sources and physical channel.

There are two fundamental issues related to information theory regarding digital
communication system

a. What is the rate at which the given source is emitting “information”?

b. What is the maximum rate of information transmission over a noisy channel?

The answer to the first question is the information theory, which deals with the
definition, measurement and modeling of information. The answer to the second
question is given by channel capacity theorem.

4.1.1. Information

A message is a sequence of symbols intended to reduce uncertainty of the receiver. If

the sequence of symbols does not change the uncertainty level of the receiver then the
message does not contain any information.

Example 4.1:

You are planning to go to Biratnagar during summer vacation. You called your friend
in Biratnagar to know the weather conditions of Biratnagar. Assuming that you
received the following message regarding the weather condition in Biratnagar in
summer

(a). It is sunny and hot.

(b). It is cold

1|P a g e
[CHAPTER-4] [Digital Communication] Information theory

(c). Snow is falling

The Information content among these three messages are different. From the past
experience about the weather condition of Biratnagar, you know that in summer
Biratnagar is sunny and hot. Therefore, the reader will hardly notice the first headline
(a) as it contains no information.

The information content in message (b) is relatively high than that in (a) because you
do not expect, in general, a cold weather during summer in Biratnagar, but may happen
rarely.

Message (c) contains highest level of information (of course among the three possible
messages you could have received) as snow during summer and in Biratnagar is almost
next to impossible.(but still could happen).

Therefore, the information content of any message is closely related to the past
knowledge of the occurrence of event and the level of uncertainty it contains with
respect to the recipient of message. The same message, if given to a person living in
North pole and have never heard of Biratnagar and have no plan to go there, would
have contained almost zero information.

Thus in general "on an intuitive basis, the amount of information received from the
knowledge of occurrence of an event is related to the probability or likelihood of
occurrence of the event".

In other words the message related to an event least likely to occur (with the
knowledge of occurring of such event in the past) contains more information.

The communication system can never be described in the deterministic sense, it can be
considered to be statistical in nature. To describe a communication system completely
we have to use its unpredictable or uncertain behaviour.

It can be easily understand by example that each transmitter transmit the information
randomly, we cannot predict which message will be transmitted the next moment. But
we know the probability of transmitting a particular message.

So to define a system completely, we need statistical study of system and statistical

study of system is performed with the help of concept of probability.

2|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Information content of a symbol

The information source can be classified as having memory or being memoryless. A

source with memory is one for which a current symbol depends on previous symbols.
A memoryless source is one for which each symbol produced is independent of the
previous symbols.

For our study we assume that the source emitted by the source during successive
signaling intervals is statistically independent (i.e., memoryless).

Let us consider a discrete memoryless source (DMS) q possible message represented

by x1 , x 2 ,...., x q  with probabilities of occurrence p1 , p2 ,...., pq  such that

p1  p 2  ....  p q  1 (4.1)
let I ( xi ) be the amount of information content in the i th message. Based on intuition,
for I ( xi ) to represent information content of x i message, the following condition
should be met.

I ( xi )  I ( x j ), if pi  p j (4.2)
Eq.(4.2) means the information content of an event with less probability is higher than
that of with high probability of occurrence.

I ( xi )  0 if pi  1 (4.3)
In Eq.(4.3) the information content in an event with the probabilty of occurrence near
to unity is near to zero. The example is the message to a person living in Earth that
"Sun rises from the east".

I ( xi )  1 if pi  0 (4.4)
The example for Eq.(4.4) is the information content in the message about an event
which is almost impossible. "Sun rises from the west".

I ( xi )  0 when 0  pi  1 (4.5)
Eq.(4.5) means that any message should contain some information i.e., I ( xi ) should be
non-negative.

3|P a g e
[CHAPTER-4] [Digital Communication] Information theory

I ( xi and x j )  I ( xi x j )  I ( xi )  I ( x j ) (4.6)
Eq.(4.6) means, if x i and x j are two independent messages coming from the same
source, then the total information received from two message is equal to the sum of
information contents in each message. For example, in weather forecast from Radio
Nepal you hear "it will be rainy today and sunny tomorrow". There are two messages
in it and if weather of today does not affect the weather of tomorrow, then the total
information in this message is equal to sum of the two individual informations.

To satisfy all the conditions mentioned above, we could relate I ( xi ) and x i in the
following manner.

1 (4.7)
I ( xi )  logb   logb pi
pi
The unit of I ( xi ) depends upon the base b assigned to log.

 base b is 'e', i.e., logarithm is natural (ln). The unit of information is 'nat'
 base b is 10, the unit is Hartley or decit.
 base b is 2, the unit is bit

If two binary digits occur with equal probability and are correctly detected at the
receiving end, then the information content in each digit is 1 bit.

(4.8)
I 0 or 1   log 2
1
 1 bit
2

4.1.2. Entropy or Average information content

The information source is usually transmitting a long sequence of symbols. Thus,

instead of define the information content of individual message symbol, it is more
practical to determine the average information content of sequence of symbols.

Let the a source emit one of k possible symbols (long independent sequence of
symbols) x1, x2, xk in statistically independent sequence (i.e., probability of
occurrence of xi, does not depend upon previous occurrence of xh or future occur of xj )
with probabilities p1. p2,... pk respectively. Now, the information content of individual

4|P a g e
[CHAPTER-4] [Digital Communication] Information theory

symbol is discrete and random in nature (i.e., cannot be determined), that takes on the
value I(x0), I(x1),…., I(xk-1). The mean value of information content I(xk) is called
Entropy and is denoted by H(X). Where, X is the discrete random variable.

H ( X )  E[ I ( xk )] (4.9)
K 1 (4.10)
H ( X )   pk I ( xk )
k 1
K 1 1 (4.11)
H ( X )   p k log 2
k 1 pi
Entropy is the measure of the average information content per source symbol. If the
information is in bits then the unit of entropy is bits/symbol

4.1.3. Information Rate

If the rate at which source emits symbols is r, than the information rate R of the source
is given by

R  r H ( X ) bit / sec (4.12)

Here R is the information rate, H(X) is entropy or average information and r is the rate
at which symbols are generated.

4.1.4. Shannon-Hartley channel capacity theorem

Channel capacity is concerned with the information handling capacity of a given

channel. It is affected by:

 The attenuation of a channel which varies with frequency as well as channel length.
 The noise induced into the channel which increases with distance.
 Non-linear effects such as clipping on the signal.

Some of the effects may change with time e.g. the frequency response of a copper
cable changes with temperature and age. Obviously, we need a way to model a channel
in order to estimate how much information can be passed through it. We can
compensate for non linear effects and attenuation but it is extremely difficult to remove
noise.

The highest rate of information that can be transmitted through a channel is called the
channel capacity, C.

5|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Shannon's Channel Capacity Theorem (or the Shannon-Hartley Theorem) states that:

 S (4.13)
C  B log 2 1   bits / sec
 N
Where C is the channel capacity, B is the channel bandwidth in hertz, S is the signal
power and N is the noise power (i.e., N0B with N0/2 being the two sided noise PSD).

Note: S/N is the ratio watt/watt not dB.

Implication of Theorem:

1. Indicates the upper limit of data transmission for reliable communication.

A designer can estimate C for required SNR and B for reliable communication.
Shannon’s Channel Coding Theorem states that, if the information rate, R (r H bits/s) is
equal to or less than the channel capacity, C, (i.e. R < C) then there is, in principle, a
coding technique which enables transmission over the noisy channel with no errors.

The inverse of this is that if R > C, then the probability of error is close to 1 for every
symbol.

2. Trade off between B and SNR for given C

The channel capacity is limited by various extraneous factors that the designer has no
control over. For example the maximum frequency that can be transmitted over a pair
of cable is limited by its construction. But the other two parameters B and SNR are in
hand of the designer. Bandwidth of the signal can be compressed and the SNR can be
increased by increasing signal power or by introducing low noise devices. Therefore ,
the designer can trade-off between B and SNR in optimal way for given C.

Example: A signal with the data rate R=10,000 bits/sec is required to transmit over a
channel with limited bandwidth of B=3000 Hz. Theoretically, the absolute minimum
bandwidth required for transmission of the above data is 10,000/2 = 5000 Hz (i.e.,
B=R/2). Now for transmission of R bits/sec, the minimum channel capacity should also
be equal to this value, i.e., Cmin=R. In this case the required SNR will be.

SNR  2 (C / B )  1  2 (10000 / 3000)  1  9

6|P a g e
[CHAPTER-4] [Digital Communication] Information theory

This shows that the signal power must be 9 times higher than the noise power. But now
if we consider a channel with B=10000 Hz, then the required SNR is

SNR  2 (C / B )  1  2 (10000 / 10000)  1  1

It means that, is the bandwidth B is higher, same quality of signal transmission can be
achieved with less power. In other words bandwidth compression from 10,000 to 3,000
Hz is possible but at the cost of increasing signal power by the factor of 9.

3. Bandwidth Compression

Shannon channel capacity theorem indicates that it is possible to transmit signal with
upper frequency fmax through a channel having bandwidth less than fmax:

Example:

Let a signal x(t) have upper frequency limit of fmax. Let us sample x(t) at 3fmax . Then
the data rate will be

R  nf s  3nf max (4.14)

where n is the number of bits per sample

Now if channel bandwidth is

B=fmax/2
then for C≥R and for n=6, the required SNR would be:
SNR  2 (C / B )  1  2 (36 f max /( f max / 2))  1  2 36  1  6.8  1010  7  1010
If we increase signal power by 7×1010 times in comparison to the noise power we can
transmit a signal through a channel having bandwidth equal to half of the signal
bandwidth. This is although possible, but very impracticable.

As the noise in the channel tends to zero (Noise→0), the value of SNR will tend to
infinity (SNR→∞). Subsequently, the channel capacity C will tend to infinity (C→∞).
This means that the noiseless channel has an infinite capacity. This type of channel is
referred to as ideal channel.

7|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Theoretical limits of Shannon's channel capacity theorem

It can be seen from Eq. that if channel bandwidth B=∞, channel capacity C=∞.
However, noise power is proportional to the bandwidth (i.e., the noise signal
considered is a white noise with a uniform power density spectrum over the entire
frequency range). Therefore, as the bandwidth B increases, noise N also increases and
hence the channel capacity remains finite at Cmax even if B=∞.

If N0/2 is the noise power density, then we have N=N0B, and

 S  (4.15)
C  B log 2 1   bits / sec
 N0 B 

lim lim S N0 B  S  (4.16)

C log2 1   bits / sec
B B   N0 S  N0 B 

N0 B (4.17)
lim S lim  S  S
C log2 1   bits / sec
B B   N0  N0 B 

The above limit may be found with the help of the following standard expression:

lim (4.18)
log 2 1  x   log 2 e  1.44
1/ x

x0
S
Therefore, replacing by x we have
N0B

N0 B (4.19)
lim Slim  S  S
C log2 1  
B B   N0  N 0 B 
lim S lim (4.20)
C log2 1  x 
1/ x

B N0 x  0

8|P a g e
[CHAPTER-4] [Digital Communication] Information theory

lim S (4.21)
C log2 e
B N0

lim S (4.22)
C  C max 1.44
B N0
So as the bandwidth goes to infinity the capacity goes to 1.44S/N0, i.e., it goes to a
finite value.

Baseband data communication

In this topic we study the transmission of digital data over a baseband channel.
Baseband digital communication system refers to a system in which transmission and
reception of digital signals over band-limited channel is accomplished without
employing carrier modulation (band pass) technique

Input data Output data

sequence sequence
Transmitter Channel Receiver

Baseband signal
processing

Fig. 4.1. Baseband digital communication

Digital data have a broad spectrum with significant low-frequency content. Baseband
transmission of digital data therefore requires the use of low-pass channel with
bandwidth large enough to accommodate the essential frequency content of the data
stream. Typically, however, the channel is dispersive i.e., its frequency response
deviates from that of an ideal low-pass filter. Thus, the received pulse is affected
somewhat by adjacent pulses, thereby giving rise to a common form of interference
called intersymbol interference (ISI).

Intersymbol Interference is a major source of bit errors in the reconstructed data stream
at the receiver output. To avoid ISI certain control measures have to be implemented.
One of the method is the use of discrete pulse modulation, in which the amplitude,

9|P a g e
[CHAPTER-4] [Digital Communication] Information theory

duration, or position of the transmitted pulses is varied in discrete manner in

accordance with the given data stream. However for the baseband transmission of
digital data, the use of discrete pulse-amplitude modulation (PAM) is one of the most
efficient schemes in terms of power and bandwidth utilization. We begin the study by
considering the case of binary data.

Baseband data communication system using PAM have the following functional
blocks shown in Fig.4.2.

Fig.4.2 Block diagram of Baseband binary data transmission system

The input binary sequence {bk} consists of symbols 1 and 0, each of duration Tb . This
sequence is applied to a pulse generator, producing the discrete PAM signal
 (4.23)
x(t )   a k p g (t  kTb )
k  

Where p g (t ) denotes the basic pulse whose amplitude a k depends upon the input data
sequence as

 1 if symbol bk is 1 (4.24)
ak  
 1 if symbol bk is 0
And is normalized such that

10|P a g e
[CHAPTER-4] [Digital Communication] Information theory

pg (0)  1 (4.25)
Signal x (t ) is passed through a transmission filter of impulse response ht (t ) . The
output further modified as it passes through the channel of impulse response hc (t ) . In
addition, the channel adds random noise to the signal at the receiver input. The channel
output is then passed through a receive filter of impulse response hr (t ) . The receive
filter output is written as

y (t )    a k p r (t  kTb   d )  n(t ) (4.26)

k

Where  is the scaling factor,  d is delay introduced in the system and n (t ) is noise.

The pulse p r (t ) is normalized such that

pr (0)  1 (4.27)
The resulting filter output y(t) is sampled synchronously with the transmitter, with the
sampling instants being determined by a clock or timing signal that is usually extracted
from the receive filter output. Finally, the sequence of samples thus obtained is used to
reconstruct the original data sequence by means of a decision device. Specifically, the
amplitude of each sample is compared to a threshold  . If the threshold  is
exceeded, a decision is made in favor of symbol 1. If the threshold  is not exceeded,
a decision is made in favor of symbol 0.

For simplicity of further analysis we assume that  d  0 and that the channel is
noiseless, i.e., n(t )  0

Then in frequency domain the received pulse can be expressed as the response of the
cascaded connection of the transmitting filter, the channel, and the receiving filter,
which is produced by the pulse p g (t ) applied to the input of this cascade connection.
Therefore, we may relate p g (t ) and p r (t ) in the frequency domain by

Pg ( f )  Pr ( f ) H T ( f ) H C ( f ) H R ( f ) (4.28)
Where Pg(f) and Pr(f) are the Fourier transforms of pg(t) and pr(t), respectively. HT(f),
HC(f) and HR(f) are the transfer functions of transmitting filter, channel and receiving
filter respectively.

11|P a g e
[CHAPTER-4] [Digital Communication] Information theory

The receiving filter output y(t) is sampled at time t=mTb is

 (4.29)
y (t  mTb )  a m    bk p r (mTb  kTb )
k  
k m
 (4.30)
y (t  mTb )  a m    a k p r [( m  k )Tb ]
k  
k m

In Eq.(4.30), the first term a m is the mth decoded bit and the second term represents
the residual effect of all other transmitted bit on the decoding of the mth bit; this
residual effect is called intersymbol interference (ISI).

ISI arises due to dispersion of pulse shape by the filters and channel. Therefore, one of
the major task of the system designer is to optimally design transmitting and receiving
filters and the shape of the basic pulse to minimize ISI.

Nyquist’s criterion for distortionless baseband binary transmission

Typically, the transfer function of the channel and the transmitted pulse shape are
known. However, as stated above, the problem is to determine the transfer function of
the transmitter and receiving filters so as to reconstruct the transmitted data sequence
{bk}. The receiver does this by extracting and then decoding the corresponding
sequence of weights, {ak}, from the output y(t).at some time t = mTb. The decoding
requires that the weighted pulse akpr(mTb - kTh) for k=m be free from ISI due to the
overlapping tails of all other weighted pulse represented by k  m . This, in turn,
requires that we control the received pulse pr(t), as shown by Eq.(4.31)

1 m  k (4.31)
p r (mTb  kTb )  
0 m  k
or in general,

1 i  0 (4.32)
p r (iTb )  
0 i  0
Where, integer i=m-k and pr(0)=1. If pr(t) satisfies the condition of Eq.(4.31), the
receiver output, given by Eq.(4.30), simplifies to

y (t  mTb )  a m (4.33)

12|P a g e
[CHAPTER-4] [Digital Communication] Information theory

which implies zero ISI. Hence, the condition of Eq.(4.32) assures perfect reception in
the absence of noise.

Frequency Domain Analysis

From a design point of view, it is informative to transform the condition of Eq. 6.23
into the frequency domain. Consider then the sequence of samples {pr(nTb)}, where n
= 0, ±1, ±2, . . . . .From chapter 2 on the sampling process for a low-pass function, we
know that sampling in the time domain produces periodicity in the frequency domain.
Thus, from Eq.(2.3) in chapter 2 we may write,
 (4.34)
P ( f )  Rb  Pr ( f  nRb )
n  

Where Rb=1/Tb is the bit rate; P ( f ) is the Fourier transform of an infinite periodic
sequence of delta function of period Tb , and whose strengths are weighted by the
receptive sample values of p(t). That is, P ( f ) is given by

 
P ( f )     p r (iTb ) (t  iTb )exp(  j 2ft )dt
(4.35)
  i  

Let the integer i=m-k. Then, m=k corresponds to i=0, and likewise m  k corresponds
to i  0 . Accordingly, imposing the condition of Eq.(4.31). on the sample values of
p(t) in the integral of Eq.(4.35)., we get
  (4.36)
P ( f )    p r (0) (t ) exp(  j 2ft )dt
  i  

P ( f )  p r (0) (4.37)
Where, we have made use of the shifting property of the delta function. Since p(0)=1,
by normalization, we thus see from Eq.(4.34).and Eq.(4.37). that the condition for zero
ISI is satisfied if
 (4.38)
 Pr ( f  nRb )  Tb
n  

Eq.(4.31) in terms of the time function pr(t), or equivalently, Eq.(4.38) in terms of the
corresponding frequency function P(f), is called the Nyquist criterion for distortionless
baseband transmission in the absence of noise.

13|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Ideal Solution

From Eq.(4.38) we see that the function P(f) represents the series of shifted spectrums.
P(f) is obtained by permitting only one non-zero component in the series (i.e., for n=0).
The range of frequencies for P(f) extend from -Bo to Bo where Bo denotes half the bit
rate:

Rb
Hence, Bo 
2

That is, P(f) is specify as a rectangular spectrum given by

1  f  (4.39)
P( f )  rect 
2 Bo  2 Bo 
This is the spectrum of a signal which produces zero ISI which is shown in Fig.4.3(a).
The time domain representation of the signal is nothing but a sinc function obtained by
taking inverse Fourier Transform of Eq.(4.39) and is shown below

 1  f  (4.40)
p(t )  F 1 P( f )  F 1  rect 
 2 Bo  2 Bo 
sin(2Bo t ) (4.41)
p(t ) 
2Bo t
 , p(t )  sinc(2Bo t ) (4.42)

(a)

14|P a g e
[CHAPTER-4] [Digital Communication] Information theory

(b)
Fig.4.3 (a). Graphical representation of P(f). (b). Time-domain response

Fig.4.4 The series of sinc pulses corresponding to the sequence 1011010

Fig.4.3(b) show plot of p(t). In Fig.4.3(b), the signaling intervals and the corresponding
centered sampling instants is also shown. The function p(t) can be regarded as the

15|P a g e
[CHAPTER-4] [Digital Communication] Information theory

impulse response of an ideal low-pass filter with pass-band amplitude response 1/(2Bo)
and bandwidth Bo. The function p(t) has its peak value at the origin and goes through
zero at integer multiples of the bit duration Tb. It is apparent that if the received
waveform y(t) is sampled at the instants of time t = 0, ± Tb, ± 2Tb,. . . , then the pulses
defined by p(t  iTb ) with arbitrary amplitude  . and index i = 0, ±1, ±2, . . . , will
not interfere with each other.

This choice of pulse shape for p(t) solves the problem of ISI with the minimum
bandwidth possible. However, there are two practical difficulties that make it an
undesirable for system design:

1. The amplitude characteristic of P(f) needs to be flat from -B0 to B0, and zero
elsewhere, which is physically unrealizable because of the abrupt transitions at ±
B0.

2. Due to discontinuity of P(f) at ± B0, there is practically no margin of error in

sampling times in the receiver.

Raised Cosine Spectrum (Practical Consideration)

The practical difficulties faced in the ideal Nyquist channel can be overcome by
extending the bandwidth from Bo = Rb/2 to an adjustable value between Bo and 2Bo. In
doing so, we expand the series on the left side of Eq.(4.38)

i.e.,
 (4.43)
 Pr ( f  nRb )  Tb
n  

and retain only three terms which corresponds to n=-1, 0 and 1 and restrict the
frequency band of interest to f  Bo as shown by

1 (4.44)
P ( f )  P ( f  2 Bo )  P ( f  2 Bo )   Bo  f  Bo
2 Bo

16|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Now, we may devise several band-limited functions that satisfy Eq.(4.44). One of the
function that match many desirable features is a raised cosine spectrum. The spectrum
characteristic of which consists of a flat portion and a roll-off portion that has a
sinusoidal form, expressed mathematically as

 1 (4.45)
 ( flat) f  f
2 Bo

 1    ( f  f1 )  
P( f )   1  cos   (rolloff ) f1  f  2 Bo  f1
 4 Bo   2 Bo  2 f1  
 0 f  2 Bo  f1


The frequency f1 and bandwidth Bo are related by

f1 (4.46)
  1
Bo
The parameter  is called the rolloff factor, which indicates the excess bandwidth
over the ideal solution Bo. For  = 0, that is, f1=Bo, we get the minimum bandwidth
solution as described in ideal case.

The time response p(t), that is, the inverse Fourier transform of P(f), is defined by

cos(2Bo t ) (4.47)
p(t )  sinc(2 Bo t )
1  16 Bo t
2 2 2

The normalized frequency response of the raised cosine function is obtained by

multiplying P(f) by 2Bo, and is shown plotted in Fig. 4.5(a) for three values of a
namely, 0, 0.5, and 1. The corresponding time response p(t) is plotted in Fig.4.5(b).

17|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Fig.4.5 Responses for different roll-off factors (a) Frequency response. (b) Time
response

This function consists of the product of two factors: the factor sinc(2Bot) associated
with the ideal filter, and a second factor that decreases as 1/|t|2 for large |t|. The first
factor ensures zero crossings of p(t) at the desired sampling instants of time t = iTb
with i an integer (positive and negative).

The second factor reduces the tails of the pulse considerably below that obtained from
the ideal low-pass filter, so that the transmission of binary waves using such pulses is

18|P a g e
[CHAPTER-4] [Digital Communication] Information theory

relatively insensitive to sampling time errors. In fact, the amount of ISI resulting from
this timing error decreases as the roll-off factor  is increased from zero to unity.

For the special case of  = 1 (i.e., f1=0) is known as the full-cosine roll-off
characteristics, for which the frequency response of Eq. simplifies to

 1   f  (4.48)
 1  cos , 0  f  2 Bo
P( f )   4 Bo   2 Bo 
 0, f  2 Bo


the function p(t) simplifies to

sinc(4 Bo t ) (4.49)
p(t ) 
1  16Bo2 t 2

General Observation

1. For α=0.5 and 1, the characteristics of P(f) changes gradually with respect to
frequency. Hence, is practically realizable.
2. There are zero crossings at t= ±3Tb/2, ±5Tb/2,... in addition to the usual zero
crossings at the sampling times t = ±Tb, ±2Tb),......
3. For α=0, the bandwidth requirement is maximum equal to 2Bo.

Correlative coding

In the previous schemes, we have treated ISI as an undesirable phenomenon that

produces degradation in system performance.

Data rate Bandwidth ISI Condition

Rb Rb/2 Zero Ideal (unrealizable)
Rb Rb Zero Raised cosine
Table.4.1 Data rate and bandwidth comparison for ideal and raised cosine.

19|P a g e
[CHAPTER-4] [Digital Communication] Information theory

It is possible to achieve a bit rate of Rb bits per second in a channel of bandwidth Rb/2
hertz by adding ISI to the transmitted signal in a controlled manner. Such schemes are
called correlative coding or partial-response signaling schemes. As ISI introduced into
the transmitted signal is known, its effect can be compensated at the receiver. Thus,
correlative coding is a practical means of achieving the theoretical maximum signaling
rate of Rb bits per sec in a bandwidth of Rb/2 hertz.

Duobinary Signaling

Duobinary signaling describes the basic idea behind correlative coding, where “duo”
implies doubling of the transmission capacity of a binary system.

Consider a binary input sequence {bk} consisting of uncorrelated binary digits each
having duration Tb seconds, with symbol 1 represented by a pulse of amplitude +1 volt,
and symbol 0 by a pulse of amplitude -1 volt. When this sequence is applied to a duo
binary encoder, it is converted into a three-level output, namely, -2, 0, and +2 volts.
This transformation is produced according to scheme shown in Fig. 4.6.

Duobinary conversion filter H(f)

Input binary
sequence Ideal channel Output
 Sequence
{bk} Hc(f)
Sample at {ck}
time
{bk-1} t=kTb
Delay
Tb

Fig. 4.6 Duobinary signaling scheme

The binary sequence {bk} is first passed through a simple filter involving a single delay
element. The digit ck at the duobinary coder output is the sum of the present binary
digit bk and its previous value bk-1, as shown by

ck  bk  bk 1 (4.50)
Such that

20|P a g e
[CHAPTER-4] [Digital Communication] Information theory

  2 if bk and bk 1 are both 1 (4.51)


c k   0 if bk and bk 1 are different
  2 if b and b are both 0
 k k 1

The effects of the transformation described by Eq.(4.50) is to change the input

sequence {bk} of uncorrelated binary digits into a sequence {ck} of correlated digits.
This correlation may be viewed as introducing intersymbol interference into the
transmitted signal in an artificial manner. However, this intersymbol interference is
under the designer’s control, which is the basis of correlative coding.

An ideal delay element, producing a delay of Tb seconds, has the transfer function
exp(  j 2fTb ) , so that the transfer function of the simple filter shown in Fig.4.6 is
1  exp( j 2fTb ) . Hence, the overall transfer function of this filter connected in
cascade with the ideal channel Hc(f) is

H ( f )  H c ( f )[1  exp( j 2fTb )] (4.52)

H ( f )  H c ( f )[exp( jfTb )  exp(  jfTb )] exp( jfTb ) (4.53)
H ( f )  2 H c ( f ) cos(fTb ) exp( jfTb ) (4.54)
For an ideal channel of bandwidth Bo=Rb/2, we have

1 f  Rb / 2 (4.55)
Hc ( f )  
0 otherwise
Thus the overall frequency response has the form of a half-cycle cosine function, as
shown by

2 cos(fTb ) exp(  jfTb ) f  Rb / 2 (4.56)

H( f )  
 0 otherwise
for which the amplitude and phase response are as shown in Fig.4.7(a) and Fig.4.7(b),
respectively. An advantage of this frequency response is that it can be easily
approximated in practice.

21|P a g e
[CHAPTER-4] [Digital Communication] Information theory

(a) (b)
Fig.4.7. Frequency response of duobinary conversion filter. (a) Amplitude response.
(b) Phase response.

The corresponding value of the impulse response consists of sinc pulse, time-displaced
by Tb seconds, as shown by

sin(t / Tb ) sin[ (t  Tb ) / Tb ] (4.57)

h(t )  
t / Tb  (t  Tb ) / Tb
sin(t / Tb ) sin(t / Tb ) (4.58)
h(t )  
t / Tb  (t  Tb ) / Tb
Tb2 sin(t / Tb ) (4.59)
h(t ) 
t (Tb  t )

Fig. 4.8 Impulse response of duobinary conversion filter

22|P a g e
[CHAPTER-4] [Digital Communication] Information theory

The above Fig.4.8 shown that h(t) has two distinguishable values at the sampling
instants +Tb and -Tb.

Detection

The original data (bk) may be detected from the duobinary-coded sequence (ck) by
subtracting the previous decoded binary digit from the currently received digit ck in

accordance with Eq.(4.50). Suppose, bk represent the estimate of the original binary
digit bk, we have
  (4.60)
bk  ck  bk 1
 
Eq.(4.60) will yield correct bk only if the previous bit bk 1 was correctly decoded at
sampling instance t = (k - 1 )Tb.

Input
Modulo-2
binary
adder
sequence ak
bk

ak 1
Delay
Tb

Precoder
Output
Sequence
ak Ideal channel {ck}

Hc(f) Sample at
time
a k 1 t=kTb
Delay
Tb

Duobinary conversion filter

H(f)

Fig.4.9 A precoded duobinary scheme

23|P a g e
[CHAPTER-4] [Digital Communication] Information theory

A practical means of avoiding this error propagation is the use of precoding before the
duobinary coding (Fig.4.6), as shown in Fig. 4.9. The precoding operation performed
on the input binary sequence (bk) converts it into another binary sequence ( a k ) defined
by

a k  bk  a k 1 (4.61)
The '  ' sign in Eq.(4.61) represents module-2 addition which is equivalent to the
EXCLUSIVE-OR operation. The resulting precoder output ( a k ) is applied to the
duobinary coder, thereby producing sequence (ck) that is related to ( a k ) as follows.

ck  a k  a k 1 (4.62)

bk  1 if ck  1 volt
ck ck Decision
Rectifier
device
bk  0 if ck  1 volt

Threshold=1
Fig.4.10 Detector for recovering original binary sequence from the precoded duobinary
coder output.

It is assumed that symbol 1 at the precoder output in Fig.4.9 is represented by +1 volt

and symbol 0 by -1 volt. Then if mth bit bm is 0 then a m  am1 from Eq.(4.61). It is
evident that for bm=0, cm will be either 2 or -2. Similarly, if bm=1, then am is the
complement of am 1 i.e., am and am 1 will always be different resulting in cm=0.

Therefore, we find that

 2 volts, if a k and a k 1 are both 1 (4.63)


ck   0 volts, if a k and a k 1 are different
  2 volts, if a and a are both 0
 k k 1

24|P a g e
[CHAPTER-4] [Digital Communication] Information theory

From Eq.(4.63), setting the threshold level at 1 and -1 we can correctly detect the
original input binary sequence {bk} from {ck}.

symbol 0 if c k  1 volt (4.64)

bk  
 symbol1 if c k  1 volt
A block diagram of the detector is shown in Fig.4.10. The detector does not require the
knowledge of any input sample other than the present sample. Hence, error
propagation cannot occur in the detector of Fig.4.10. It can be shown from example
below that, the decoding of present symbol (bk) is not affected by the error in its past
value (bk-1).

Example: Assumption: a k 1 =1

Consider a input binary sequence 0010110. Now the process of duobinary encoding
and decoding is explained in the table below.

Count (k-1) k
Input Seq. (bk) 0 0 1 0 1 1 0
precoder 1 1 1 0 0 1 0 0
output (assumed
( a k =bk  a k 1 ) value)
Polar +1 +1 +1 -1 -1 +1 -1 -1
representation
of a k
DB encoder +2 +2 0 -2 0 0 -2
output
(ck= a k + a k 1 )
Decoded bit 0 0 1 0 1 1 0
using Eq.(4.64)
The same result is obtained for the assumption a k 1 =0.

Modified Duobinary Technique

The modified duobinary technique involves a correlation span of two binary digits.
This is achieved by subtracting input binary digits spaced 2Tb seconds apart, as

25|P a g e
[CHAPTER-4] [Digital Communication] Information theory

indicated in the block diagram of Fig. 4.11. The output of the modified duobinary
conversion filter is related to the sequence at its input as follows

ck  ak  ak 2 (4.65)
Here, three-level signal is generated. If a k = ±1 volt assumed previously, ck takes on
one of three values i.e., 2, 0, and -2 volts.

Input
Modulo-2
binary
adder
sequence ak
bk

ak2
Delay
2Tb

Precoder
Output
Sequence
ak + Ideal channel {ck}

- Hc(f) Sample at
time
ak2 t=kTb
Delay
2Tb

Modified Duobinary
conversion filter H(f)

Fig.4.11 Modified duobinary signaling scheme.

The overall transfer function of the tapped-delay-line filter connected in cascade with
the ideal channel, as in Fig. 4.11, is given by

H ( f )  H c ( f )[1  exp( j 4fTb )] (4.66)

H ( f )  2 jH c ( f ) sin(2fTb ) exp(2 jfTb ) (4.67)
Here, Hc(f) is as defined in Eq.(4.55). Therefore, the overall frequency response is in
the form of a half-cycle sine function, as

26|P a g e
[CHAPTER-4] [Digital Communication] Information theory

2 j sin(2fTb ) exp(  j 2fTb ) f  Rb / 2 (4.68)

H( f )  
 0 elsewhere
The corresponding amplitude response and phase response of the modified duobinary-
coder are shown in Fig.4.12 (a) and (b), respectively. The phase response in
Fig.4.12(b) does not include the constant 90° phase shift due to the multiplying factor j.
A useful feature of the modified duobinary coder is that its output has no dc
component. This property is important, as in practice many communication channels
cannot transmit a dc component.

(a)
(b)
Fig.4.12 Frequency response of the modified duobinary conversion filter (a)
Magnitude response.(b) Phase response.

The impulse response of the modified duobinary coder consists of sinc pulse that is
time-displaced by 2Tb seconds, as shown

sin(t / Tb ) sin[ (t  2Tb ) / Tb ] (4.69)

h(t )  
t / Tb  (t  2Tb ) / Tb
sin(t / Tb ) sin(t / Tb ) (4.70)
h(t )  
t / Tb  (t  2Tb ) / Tb
2Tb2 sin(t / Tb ) (4.71)
h(t ) 
t (2Tb  t )
This impulse response is plotted in Fig. 4.13, which shows that it has three
distinguishable levels at the sampling instants.

27|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Fig.4.13 Impulse response of the modified duobinary conversion filter.

In order to eliminate the possibility of error propagation in the modified duobinary

system, we use a precoding procedure similar to that used for the duobinary case. Prior
to the generation of the modified duobinary signal, a modulo-2 logical addition is used
on signals 2Tb seconds apart, as shown in Fig. 4.11.

a k  bk  a k 2 (4.72)
where (bk) is the input binary sequence and ( a k ) is the sequence at the precoder output.
Note that modulo-2 addition and modulo-2 subtraction are the same. The sequence ( a k
) thus produced is then applied to the modified duobinary conversion filter.

In the case of Fig.4.11, the output digit ck equals 0,+2, or -2 volt. ck is obtained in the
way similar to the duobinary technique. Specifically, the original sequence (bk) is
obtained at the receiver using the following decision rule:

symbol 0 if ck  1volt (4.73)

bk  
 symbol 1 if c k  1volt

M-ary signaling

In binary signaling, the output of the pulse generator can have one of two possible
levels. In M-ary signaling, the output can have one of M possible levels. In M-ary

28|P a g e
[CHAPTER-4] [Digital Communication] Information theory

system the input source emits one of M distinct symbols and each symbol is assigned a
distinct level out of M possible levels. For example, for M=4, the signal representation
can be defined as

Input symbol Representation Level Bit representation

B -3A 00
C -A 01
D +A 10
E +3A 11
Combination of two bits can be represented by one level in 4-ary signaling system. For
M=4 (i.e., 22), the combination of input bit stream in binary form can be represented by
slicing the stream into four groups of two bits (00, 01, 10, 11). Each group is then
assigned a fixed level.

+3A

00 01 +A

-A 10 11
-3A
Fig.4.14 4-ary Signaling scheme.

This signaling allows us to transmit each pair of binary digits by one 4-ary pulse.
Hence to transmit n binary digits, we need only (n/2) 4-ary pulse.

At receiving end, the decoded output is compared with present threshold values
(slicing levels) and decision is made.

In general, the information IM transmitted by an M-ary symbol is

IM=log2M bits (4.74)

The signaling rate and bandwidth requirement of M-ary

29|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Let Rs (symbols/sec or Baud) be the rate of symbols emitted by the source. Now if the
M symbols emitted are equiprobable and statistically independent, then the source
entropy will be.
M (4.75)
H ( X )   pi log2 (1 / pi )
i 1

As all symbols are equiprobable

1
p1  p 2  ...  p M 
M
Thus,

M 1 (4.76)
H (X )   log 2 ( M )
i 1 M

M (4.77)
H (X )  log 2 ( M )
M
H ( X )  log2 (M ) (4.78)
If the rate at which source emits symbols is r, than the information rate R of the source
is given by

R  r H ( X ) bit / sec (4.79)

R  Rs log2 M bit / sec (4.80)
The signaling interval duration Ts=1/Rs, is same for both binary and M-ary systems.
Therefore, the absolute minimum bandwidth required to transmit Rs log2 M bits/sec of
information is Rs/2 Hz.

Under similar conditions (i.e., Tb=Ts), the signaling rate for binary system is

Rb  Rs log2 2  Rs (4.81)

And the bandwidth is also Rs/2. It means M-ary signaling can transmit data log2 M
times faster than binary system under similar conditions. The price paid for higher
speed (or subsequently less bandwidth compared to binary) in M-ary system is the
power required to transmit M-ary signal. Thus there exists a trade-off between power
and bandwidth.

30|P a g e
[CHAPTER-4] [Digital Communication] Information theory

M-ary is more complex since it requires (M-1) comparators at the receiving end for
threshold detection.

Eye Diagram

Eye diagram is a practical way to study the ISI and its effect on PCM. Eye pattern is
obtained on cathode ray oscillator (CRO) by applying received signal to vertical input
and sawtooth wave at the transmission symbol rate (i.e., R=1/Tb) to horizontal input.
This resulting oscilloscope display is called an eye pattern because of its resemblance
to the human eye for binary waves as shown in Fig.4.15. The interior region of the eye
pattern is called the eye opening.

Fig.4.15 (a) Distorted binary wave. (b) Eye pattern

An eye pattern provides a great deal of information about the performance of the
pertinent system, as shown in Fig.4.16.

31|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Fig.4.16 Interpretation of eye pattern.

The information provided by the eye pattern is

1. The width of the eye opening defines the time interval over which the received
wave can be sampled without error from intersymbol interference. It is apparent that
the preferred time for sampling is the instant of time at which the eye is open widest.

2. The sensitivity of the system to timing error is determined by the rate of closure of
the eye as the sampling time is varied.

3. The height of the eye opening, at a specified samplingtime, defines the margin over
noise.

When the effect of intersymbol interference is severe, traces from the upper portion of
the eye pattern cross traces from the lower portion, with the result that the eye is
completely closed. In such a situation, it is impossible to avoid errors due to the
combined presence of intersymbol interference and noise in the system, and a solution
has to be found to correct for them.

Line coding

32|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Binary 1’s and 0’s, in PCM signaling, may be represented in various serial-bit
signaling formats called line codes. The simplest is to represent ‘1’ by a square pulse
of A volt and ‘0’ by 0 volt. But the simplest is not always good enough. Also, a long
sequence of ‘0' may appear as a loss of transmission. In order to take care of these and
many other requirements, the symbols are transformed in to various different wave
shapes by the process named line coding. Different wave shapes result in different
spectrum, suiting different needs. Hence in a way, line codes are also spectrum shaping
codes. Thus line coding is mapping of binary information sequence into the digital
signal that enters the channel

Necessary properties of line code

1. Self synchronization - It should be possible to recover the clock pulse from the
received data. Clock should not be lost even in case of a long sequence of ‘0'. Self-
synchronization, where the timing information is extracted from the received signal
itself
2. Low probability of bit error - It should be convenient to design a receiver, which
receives the specific line code and results a low probability of bit error.
3. PSD - Spectrum of the line code should suit the physical medium.
4. Bandwidth - Bandwidth of the line coded signal should be low.
5. No DC - DC power content should be ideally zero to enable AC coupling.
6. Low frequency power - Power at very low frequency should be as low as possible.
7. To suit channel coding - Line code should be such that subsequent coding for error
detection/correction is easy.
8. Power efficiency - Required transmission power should be small.
9. Transparency –a line code is transparent if any bit pattern does not affect the
accuracy of the timing. A transmitted signal would not be transparent if there are a
long series of 0's which would cause an error in the timing information.

Types of Line coding

Unipolar Signaling

Unipolar signaling (also called on-off keying, OOK) is the type of line coding in which
one binary symbol (representing a 0 for example) is represented by the absence of
pulse (i.e., a SPACE or OFF) and the other binary symbol (denoting a 1) is represented
by the presence of a pulse (i.e., a MARK or ON).

33|P a g e
[CHAPTER-4] [Digital Communication] Information theory

There are two common variations of unipolar signaling

i. Unipolar Non-Return to Zero (NRZ) Signaling

In this line code, symbol 1 is represented by transmitting a pulse of amplitude A for the
duration of the symbol, and symbol 0 is represented by switching off the pulse, as in
Fig.4.17. The unipolar NRZ line code is also referred to as ON-OFF signaling. Each
ON pulse is equal to the duration of Tb of the symbol slot.

Binary data
1 0 1 0 1 1 1 1 1 0
A

0
Tb

Fig.4.17 Unipolar Non-Return to Zero (NRZ) signaling.

Advantages

 Simplicity in implementation.
 Doesn’t require a lot of bandwidth for transmission.

Disadvantages

 Presence of DC level (indicated by spectral line at 0 Hz in Fig.)

 Contains low frequency components. Causes “Signal Droop”.
 Does not have any error correction capability.
 Does not possess any clocking component for ease of synchronization.
 Is not Transparent. Long string of zeros causes loss of synchronization.

34|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Fig.4.18 PSD of Unipolar NRZ.

When Unipolar NRZ signals are transmitted over links with either transformer or
capacitor coupled (AC) repeaters, the DC level is removed converting them into a
polar format.

The continuous part of the PSD is also non-zero at 0 Hz (i.e. contains low frequency
components) as shown in Fig.4.18. This means that AC coupling will result in
distortion of the transmitted pulse shapes. AC coupled transmission lines typically
behave like high-pass RC filters and the distortion takes the form of an exponential
decay of the signal amplitude after each transition. This effect is referred to as “Signal
Droop” and is illustrated in Fig.4.19.

35|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Fig.4.19 Distortion (Signal Droop) due to AC coupling of unipolar NRZ signal

Disadvantages of on-off signaling are the waste of power due to the transmitted DC
level and the fact that the power spectrum of the transmitted signal does not approach
zero at zero frequency.

ii. Unipolar Return to Zero (RZ) Signaling

In this line code, symbol 1 is represented by a rectangular pulse of amplitude A for

duration half of symbol width (i.e.,Tb/2) and symbol 0 is represented by transmitting
no pulse, as illustrated in Fig.4.20.

Binary data
1 0 1 0 1 1 1 1 1 0
A

0
Tb

Fig.4.20 Unipolar Return to Zero (RZ) signaling.

Advantages

36|P a g e
[CHAPTER-4] [Digital Communication] Information theory

 Simplicity in implementation.
 Presence of spectral line at symbol rate which can be used as symbol timing
clock signal.

Disadvantages

 Presence of DC level (indicated by spectral line at 0 Hz in Fig.)

 Continuous part is non-zero at 0 Hz. Causing “Signal Droop”.
 Does not have any error correction capability.
 Occupies twice as much bandwidth as Unipolar NRZ.
 Is not Transparent. Long string of zeros causes loss of synchronization.

Fig.4.21 psd of Unipolar RZ.

An attractive feature of the unipolar RZ line code is the presence of delta functions at
f  0,1 / Tb in the power spectrum of the transmitted signal; the delta functions can be
used for bit-timing recovery at the receiver. However, its disadvantage is that it
requires 3 dB more power than polar RZ signaling for the same probability of symbol
error.

Thus it can be concluded that neither variety of unipolar signals is suitable for
transmission over AC coupled lines.

Polar Signaling

37|P a g e
[CHAPTER-4] [Digital Communication] Information theory

In polar signaling a binary “1” is represented by a pulse g1(t) and a binary “0” by the
opposite (or antipodal) pulse g0(t)=-g1(t). Polar signaling also has NRZ and RZ form.

i. Polar Non-Return to Zero (NRZ) Signaling

In this line code, symbols 1 and 0 are represented by transmitting pulses of amplitudes
+A and -A. respectively, as illustrated in Figure 4.22. The polar NRZ line code is
relatively easy to generate. However, it has some advantages as well as disadvantages.

Binary data
1 0 1 0 1 1 1 1 1 0
A

0
Tb

-A

Fig.4.22 Polar Non Return to Zero (NRZ) signaling.

Advantages

 Simplicity in implementation.
 No DC component.

Disadvantages

 Continuous part is non-zero at 0 Hz. Causing “Signal Droop”.

 Does not have any error correction capability.
 Does not possess any clocking component for ease of synchronization..
 Is not Transparent. Long string of zeros causes loss of synchronization.

38|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Fig.4.23 psd of Polar NRZ

but its disadvantage is that the power spectrum of the signal is large near zero
frequency.

ii. Polar Return to Zero (RZ) Signaling

In this line code, symbol 1 is represented by a rectangular pulse of amplitude A for

duration half of symbol width (i.e.,Tb/2) and symbol 0 is represented by transmitting a
rectangular pulse of amplitude -A pulse for duration half of symbol width (i.e.,Tb/2), as
illustrated in Fig.4.24.

Fig.4.24 Polar Return to Zero (RZ) signaling.

Advantages

39|P a g e
[CHAPTER-4] [Digital Communication] Information theory

 Simplicity in implementation.
 No DC component.

Disadvantages

 Continuous part is non-zero at 0 Hz. Causing “Signal Droop”.

 Does not have any error correction capability.
 Does not posses any clocking component for ease of synchronization.
However, clock can be extracted by rectifying the received signal.
 Occupies twice as much bandwidth as Polar NRZ.

Fig.4.25 psd of Polar RZ

Bipolar RZ Signaling

Bipolar signaling is also called Alternate Mask Inversion (AMI) which uses three
amplitude levels (A,0,-A). ‘0s’ as in unipolar are represented by the absence of a pulse
and ‘1s’ (MARK or ON) are represented by alternating voltage levels of +A and –A.
Alternating the MARK level voltage ensures that the bipolar spectrum has a null at DC
and the signal droop on AC coupled line is avoided. The alternating mark voltage also
gives bipolar signaling a single error detection capability. Like the Unipolar and Polar
cases, Bipolar also has NRZ and RZ variations.

i. Bipolar/AMI NRZ

40|P a g e
[CHAPTER-4] [Digital Communication] Information theory

In this line code, ‘0s’ are represented by the absence of a pulse and ‘1s’ (MARK or
ON) are represented by alternating voltage levels of +A and –A with each pulse having
symbol width equal to Tb as shown in Fig.4.26.

Fig.4.26 AMI NRZ

Advantages

 No DC component.
 Occupies less bandwidth than unipolar and polar NRZ schemes.
 Does not suffer from signal droop (suitable for transmission over AC coupled
lines).
 Possesses single error detection capability.

Disadvantages

 Does not possess any clocking component for ease of synchronisation.

 Is not Transparent.

41|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Fig.4.27 psd of Bipolar/AMI NRZ

ii. Bipolar/AMI RZ

In this line code, ‘0s’ are represented by the absence of a pulse and ‘1s’ (MARK or
ON) are represented by alternating voltage levels of +A and –A with each pulse having
half symbol width (i.e., Tb/2) as shown in Fig.4.28.

Fig.4.28 Bipolar RZ

Advantages

 No DC component.
 Occupies less bandwidth than unipolar and polar RZ schemes.

42|P a g e
[CHAPTER-4] [Digital Communication] Information theory

 Does not suffer from signal droop (suitable for transmission over AC coupled
lines).
 Possesses single error detection capability.
 Clock can be extracted by rectifying (a copy of) the received signal.

Disadvantages

 Is not Transparent.

Fig.2.29 psd of Bipolar RZ

Split-Phase (Manchester Code) Signaling

In this method of signaling, illustrated in Fig.2.30, the voltage remains at one level
during the first half and moves to the other level during the second half.

A ‘1’ is positive in 1st half and negative in 2nd half. A ‘0’ is negative in 1st half and
positive in 2nd half.

1 0
+ve +ve

0 0

-ve -ve

43|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Fig.2.30 Representation of one and zero using Manchester code.

Fig.2.31 Split-Phase (Manchester Code) Signaling

The transition at the centre of every bit interval is used for synchronization at the
receiver. Manchester encoding is called self-synchronizing. Synchronization at the
receiving end can be achieved by locking on to the transitions, which indicate the
middle of the bits.

It is worth highlighting that the traditional synchronization technique used for unipolar,
polar and bipolar schemes, which employs a narrow BPF to extract the clock signal
cannot be used for synchronization in Manchester encoding. This is because the PSD
of Manchester encoding does not include a spectral line/ impulse at symbol rate (1/Tb).
Even rectification does not help.

44|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Fig.2.32 pdf of Manchester

Advantages

 No DC component.
 Does not suffer from signal droop (suitable for transmission over AC coupled
lines).
 Easy to synchronize with.
 Is Transparent.

Disadvantages

 Because of the greater number of transitions it occupies a significantly large

bandwidth.
 Does not have error detection capability.

Differential coding

Change in polarity i.e., +A becomes –A and vice versa due to error in some system
results in error in subsequent bits in Polar NRZ coding. Differential line coding
provides robustness to this type of error. In differential coding, ‘1’ is mapped as a
transition in signal level whereas, ‘0’ is mapped as no transition in signal level. The pdf
of differential code is same as NRZ scheme.

45|P a g e
[CHAPTER-4] [Digital Communication] Information theory

Differential Manchester coding

In differential Manchester encoding shown in Fig.2.33, the inversion at the middle of

the bit interval is used for synchronization but the presence or absence of an additional
transition at the beginning of the interval is used to identify the bit.

A transition means binary 0 and no transition means binary 1. Differential Manchester

encoding requires two signal changes to represent binary 0 but only one to represent
binary 1.

Binary data
1 0 1 0 1 1 1 1 1 0
A

Manchester 0

-A

Differential
0
Manchester

-A

Fig.2.33 Manchester and Differential Manchester.

Thus, differential encoder uses the presence or absence of transitions to indicate logical
value. This gives it several advantages over Manchester encoding

 Detecting transitions is often less error-prone

 Because only the presence of a transition is important rather than polarity,
differential scheme will work exactly the same if the signal is inverted.

High Density Bipolar (HDB) Signaling

46|P a g e
[CHAPTER-4] [Digital Communication] Information theory

In case of unipolar RZ, unipolar NRZ, bipolar NRZ or AMI signal, the transmitted
signal is equal to zero when a binary 0 is to be transmitted. Transmission of long
sequence of binary 0 can cause problem in synchronization at the receiver. The
solution is to add pulses when long strings of 0’s exceeding a number n are being
transmitted. This type of coding is called High Density Bipolar (HDB) coding. It is
denoted by HDBn where n=1, 2, 3,…. The most widely used HDB format is with n=3
i.e., HDB3 as shown in Fig.2.34.

Fig.2.34 HDB3 code

In HDB-3 a string of 4 consecutive zeros are replaced by either 000V or B00V. Where,
‘B’ conforms to the Alternate Mark Inversion Rule and ‘V’ is a violation of the
Alternate Mark Inversion Rule. The reason for two different substitutions is to make
consecutive Violation pulses alternate in polarity to avoid introduction of a DC
component.

The substitution is chosen according to the following rules:

1. If the number of nonzero pulses after the last substitution is odd, the substitution
pattern will be 000V.
2. If the number of nonzero pulses after the last substitution is even, the substitution
pattern will be B00V.

Advantages
 No DC component.
 Occupies less bandwidth than unipolar and polar RZ schemes.
 Does not suffer from signal droop (suitable for transmission over AC coupled
lines).

47|P a g e
[CHAPTER-4] [Digital Communication] Information theory

 Possesses single error detection capability.

 Clock can be extracted by rectifying (a copy of) the received signal.
 Is Transparent.

Disadvantage
 As HDB3 is a bipolar signal, the receiver has to distinguish between three
levels (+A, -A, and 0), instead of just two levels as in other signaling formats
previously discussed.
These characteristic make this scheme ideal for use in Wide Area Networks

Previous Exam Questions

1. Define Inter Symbol Interference (ISI)? State Nyquist Pulse Shaping criteria for
Zero ISI, Discuss Raised Cosine Pulse Shaping method of ISI reduction.
2. Briefly explain duo binary Encoding method for ISI reduction.
3. Show. that Duo-binary Encoding can be used to minimize ISI.
4. Explain any one of the practical methods of reducing ISI to zero.
5. What is ISI? Explain two practical methods of minimizing ISI.
6. Derive the expression of the signal at the input of the receiver of a baseband
DCS and based on that expression, define ISI. State and explain Nyquist Pulse
Shaping criteria for zero ISI.
7. What do you mean by duo-binary encoding? What is its importance; Explain
duo-binary encoding with example.
8. Define information and entropy. Relate message, entropy and information.
9. Define information and entropy. Calculate the upper limit of the channel
capacity as the bandwidth of the channel (B) tends to infinity.
10. Define information. What is the basic condition to be fulfilled if the
parameter “i” is to represent the amount of information contained in an
event (message) “mk”.
11. Differentiate between message and information. Derive the expression for
evaluating the entropy of source that emits M non-equioprobable symbols in
statistically independent manner.

48|P a g e
[CHAPTER-4] [Digital Communication] Information theory

12. Differentiate between entropy and information. Define entropy and derive the
expression for evaluating the entropy of source emitting symbols in statistically
independent manner.
13. Define entropy and derive the expression for evaluating the entropy of source
emitting symbols in statistically independent manner. A discrete source emits 3
symbols with probabilities are 1/3,1/6 and ½. Calculate Entropy.
14. Differentiate between message and information. A discrete source emits one of
six possible symbols per microsecond in statistically independent manner. The
symbol probabilities are 1/4, 1/4. 1/4, 1/8, 1/16 and 1/16 respectively.
Calculate: (a) Symbol Rate (b) Entropy (c) information Rate.
15. A discrete source emits one of 5 symbols per milliseconds in statistically
independent manner. The symbol probabilities are 1/2, 1/4, 1/8. 1/16 and 1/16
respectively. Calculate (a) Symbol Rate (b) Entropy (c) Information Rate.
16. A discrete source emits one of 6 symbols per 10 μs in statistically independent
manner. The symbol probabilities are 1/4,. 1/4, 1/4,.1/8, 1/16 and 1/16
respectively. Calculate (a)symbol Rate (b) Entropy (c) Information Rate.
17. Define information. A memory less discrete source emits one of 3 possible
symbols per microsecond in statistically independent manner. The symbol
probabilities are 1/4, 1/4, and 1/2 respectively. Calculate (a) Symbol rate (b)
Entropy (c) Information rate.
18. A signal of bandwidth 4.5 kHz is sampled at the double rate given by Nyquist
Rate and the signal is quantized in 8 levels with probabilities of occurrence of
the level are 0.1, 0.15, 0.15, 0.05, 0.2, 0.05, 0.18. 0.12. Calculate (a) minimum
no. of bits per sample (b) Information Rate.
19. A analog signal band-limited to 10 kHz is sampled at Nyquist Rate and
quantized in 8 levels with probabilities of 1/4. 1/5. 1/5, 1/10,.1/10, 1/20, 1/20
and 1/20 respectively. Calculate: (a) Entropy (b) Information Rate.
20. State and discuss Shannon’s Channel Capacity Theory, Based on this theory
discuss how the bandwidth and SNR can be trade-off for given channel
capacity.
21. State and discuss Shannon's Channel Capacity Theory. Discuss the implications
and theoretical limits of this theory. With examples, discuss how the bandwidth
can be traded with power for given channel capacity.
22. Write short Notes on:
• Eye Diagram

49|P a g e
[CHAPTER-4] [Digital Communication] Information theory

• Performance evaluation of DCS using Eye Diagram

23. Represent binary sequence 1011001010 in Polar NRZ, Polar RZ, Manchester
and AMI codes.
24. Differentiate between message and information? A discrete source is emitting
one of 5 possible symbols per 10 Microsec. The probabilities are 1/2, 1/4, 1/8,
1/16, and 1/16. Find (a) Symbol Rate (b) Source entropy (c) Information rate.
25. What is ISI? State Nyquist pulse shaping criteria for zero ISI. Explain duo-
binary encoding with example.

50|P a g e