ECT305 Analog and Digital Communication

Module 3 – Source Coding, Waveform Coding,

Sampling and Quantization

Lakshmi V.S.
Assistant Professor
Electronics & Communication Department
Sree Chitra Thirunal College of Engineering, Trivandrum
 Source Coding
• Source Coding – provides data compression

• Source coding tries to remove data redundancy and represent source output with minimum number
of code symbols

• Increases the transmission efficiency

2
 Source Coding
• Source Coding is the process by which the data (information) generated by a discrete source is
efficiently represented.

• Knowledge of statistics of the source is required for efficient source coding

• If some source symbols are more probable than others, this feature can be exploited for
generating source codes

• Assign shorter codewords to source symbols with high frequency

• Eg. Morse code (variable length code)

Department of Electronics & Communication 3

 Source Coding
• Let the source be characterized by a set of ‘q’ symbols

• Source alphabet, S= {s1, s2, …, sq}

• Let the code be formed using a combination of ‘r’ symbols from X

• Code alphabet, X= {x1, x2, …, xr}

• Coding means representing each and every source symbol of S by a sequence of symbols of X in a
one to one manner

• Codeword is any sequence of symbols from the code alphabet

• Codeword length - total number of symbols contained in the codeword

Department of Electronics & Communication 4

 Example
Source alphabet, S= {s1, s2, s3, s4}
Code alphabet, X= {x1, x2, x3, x4, x5, x6}

Source Symbols Codeword Codeword length

s1 x3 1
s2 x1x2 2
s3 x2x3x4x3 4
s4 x1x3x4x5x6 5

Department of Electronics & Communication 5

 Source Coding Theorems
Source Coding Theorem I (Lossless Source Coding Theorem)
• Let X be the ensemble of letters from a DMS with finite entropy 𝐻 𝑋 . Blocks of 𝐽 symbols from the
source are encoded into codewords of length 𝑁 from a binary alphabet. For any 𝜀 > 0, the
probability 𝑃𝑒 of a decoding failure can be made arbitrarily small if
𝑁
𝑅≡ ≥𝐻 𝑋 +𝜀
𝐽
and 𝐽 is sufficiently large.
• Conversely, if
𝑅 ≤𝐻 𝑋 −𝜀
then 𝑃𝑒 becomes arbitrarily close to 1 as 𝐽 is made sufficiently large.
This implies that the average number of bits per symbol required to encode the output of a DMS with
arbitrarily small probability of decoding failure is lower bounded by the source entropy 𝐻 𝑋 . On the
other hand, if 𝑅 < 𝐻 𝑋 , the decoding failure rate approaches 100% as 𝐽 is arbitrarily increased.

Department of Electronics & Communication 6

 Source Codes - Instantaneous Codes – Prefix Property
• Codewords generated corresponding to each source symbol should be instantaneous codes. (to
reduce delay in decoding).
• For a codeword set to be instantaneous, it should satisfy prefix property.
Prefix Property
• “A necessary and sufficient condition for a code to be instantaneous is that no complete
codeword be a prefix of some other codeword”.

Department of Electronics & Communication 7

 Optimal Codes
• An instantaneous code is said to be optimal if it has “minimum average length”, for a source with
given probability assignments for source symbols.
• For this, source symbols with higher probabilities are assigned shorter codewords.

Source Symbols s1 s2 … sq
Probabilities p1 p2 … pq
Codeword lengths l1 l2 … lq
𝑞

Average length, 𝐿 = ෍ 𝑝𝑘 𝑙𝑘
𝑘=1

Department of Electronics & Communication 8

 Source Coding Theorems
Source Coding Theorem II
• Let X be the ensemble of letters from a DMS with finite entropy 𝐻 𝑋 , and output letters 𝑥𝑘 , 1 ≤
𝑘 ≤ 𝑞, with corresponding probabilities of occurrence 𝑝𝑘 , 1 ≤ 𝑘 ≤ 𝑞. It is possible to construct a
prefix code with average length 𝐿 that satisfies the inequalities,
𝐻 𝑋 ≤𝐿 ≤𝐻 𝑋 +1
𝑞
where 𝐿 = σ𝑘=1 𝑝𝑘 𝑙𝑘 and 𝑙𝑘 , 1 ≤ 𝑘 ≤ 𝑞 denotes the length of codewords corresponding to source
symbols 𝑥𝑘 .

Department of Electronics & Communication 9

 Pulse Modulation
• In continuous-wave (CW) modulation, some
parameter of a sinusoidal carrier wave is varied
continuously in accordance with the message signal.

• In pulse modulation, some parameter of a pulse train

is varied in accordance with the message signal

• Pulse Modulation – Carrier is periodic sequence of

rectangular pulses
 Waveform Coding
• Waveform coding means encode the waveform in an efficient way.
• The use of coded pulses for the transmission of analog information-bearing signals represents a
basic ingredient in digital communication.
• Pulse code modulation (PCM) is a type of coding that is called "waveform" coding because it creates
a coded form of the original voice waveform.

11
 Applications of PCM
• PCM was introduced in the U.S. in the early 1960s when the telephone companies began converting
voice to digital for transport over intercity trunks.
• It is used in telephony and compact discs, satellite transmission systems and space communications.
 Pulse Code Modulation (PCM)
• PCM is the digital form of pulse modulation.
• Analog information signal is converted to digital form via
– Sampling - Makes the signal discrete in time.
– Quantization - Round off the discrete amplitude to one of the
available discrete levels.
– Encode - Maps the quantized values to digital words that are of
certain bits long.
• In PCM, these coded pulses are transmitted.
 Sampling
• The sampling process is usually described in the time domain.
• In sampling process, a continuous time signal is converted to
discrete time signal.
• If we sample the signal g(t) instantaneously (using delta
function at 𝑡 = 𝑛𝑇𝑠 and at a uniform rate, once every Ts seconds, Analog Signal
we obtain an infinite sequence of samples spaced Ts seconds
apart and denoted by {𝑔(𝑛𝑇𝑠 )}.
• This ideal form of sampling is called instantaneous sampling.
• 𝑔𝛿 (𝑡) is the ideal sampled signal.
∞

𝑔𝛿 (𝑡) = ෍ 𝑔(𝑛𝑇𝑠 )𝛿(𝑡 − 𝑛𝑇𝑠 ) Ts sampling period

𝑛=−∞
Instantaneously sampled version of
analog Signal
Sampling Process 14
 Sampling – Frequency Domain Representation
∞

𝑔𝛿 (𝑡) = ෍ 𝑔(𝑛𝑇𝑠 )𝛿(𝑡 − 𝑛𝑇𝑠 )

𝑛=−∞

Gδ f is the discrete-time Fourier transform of 𝑔𝛿 (𝑡)

∞ ∞ Analog Signal
𝐺𝛿 𝑓 = ෍ 𝑔 𝑛𝑇𝑠 න 𝛿 𝑡 − 𝑛𝑇𝑠 exp −𝑗2𝜋𝑓𝑡 𝑑𝑡
𝑛=−∞ −∞
∞

𝐺𝛿 𝑓 = ෍ 𝑔(𝑛𝑇𝑠 ) exp −𝑗2𝜋𝑛𝑇𝑠 𝑓

𝑛=−∞
∞ ∞

𝑔𝛿 (𝑡) = ෍ 𝑔(𝑛𝑇𝑠 )𝛿(𝑡 − 𝑛𝑇𝑠 ) ⇒ 𝐺𝛿 (𝑓) = 𝑓𝑠 ෍ 𝐺(𝑓 − 𝑚𝑓𝑠 )

𝑛=−∞ 𝑚=−∞

Instantaneously sampled version of

Ts - sampling period, fs = 1/Ts - sampling rate analog signal
15
Sampling Process
 Spectrum of Sampled Signal
• If the signal g(t) is strictly band limited, with no frequency components higher than W hertz.
• Then the Fourier transform 𝐺 𝑓 of the signal g(t) has the property that, 𝐺 𝑓 = 0, 𝑓 ≥ 𝑊 [Fig (a)]
• If sampling period 𝑇𝑠 = 1/2𝑊, then spectrum 𝐺𝛿 𝑓 of the sampled signal 𝑔𝛿 𝑡 is shown in Fig. (b).
• Uniform sampling in the time domain, results in a periodic spectrum with a period equal to the
sampling rate. ∞

𝐺𝛿 (𝑓) = 𝑓𝑠 ෍ 𝐺(𝑓 − 𝑚𝑓𝑠 )

𝑚=−∞

𝑇𝑠 = 1/2𝑊 → 𝑓𝑠 = 2𝑊

(a) Spectrum of a strictly (b) Spectrum of the sampled version of g(t) for a sampling period Ts = 1/2W.
band-limited signal g(t).
16
 Sampling Theorem
• Sampling theorem for strictly bandlimited signals of finite energy can be stated in two equivalent
parts:

1. A band-limited signal of finite energy that has no frequency components higher than W hertz is
completely described by specifying the values of the signal instants of time separated by 1/2W
seconds. (performed at the transmitter)

2. A band-limited signal of finite energy that has no frequency components higher than W hertz is
completely recovered from a knowledge of its samples taken at the rate of 2W samples per second.
(performed at the receiver)

• For a signal bandwidth of W hertz, the sampling rate of 2W samples per second, is called the Nyquist
rate; its reciprocal 1/2W (measured in seconds) is called the Nyquist interval.

17
 Sampling Theorem
• Sampling theorem states that for proper
reconstruction of the analog signal from its 𝑓𝑠 > 2𝐵 (oversampling)
sampled version, sampling rate should be at least
equal to twice the bandwidth. 𝑓𝑠 ≥ 2𝐵, where 𝐵 –
signal bandwidth
• 𝑓𝑠 < 2𝐵 leads to signal distortion 𝑓𝑠 = 2𝐵 (Nyquist rate)
(undersampling ⇒Aliasing)
• Aliasing refers to the phenomenon of a high-
frequency component in the spectrum of the
signal, seemingly taking on the identity of a lower 𝑓𝑠 < 2𝐵 (undersampling)
frequency in the spectrum of its sampled version.

18
 Sampling Theorem
• To combat the effects of aliasing in practice, we
may use two corrective measures: 𝑓𝑠 > 2𝐵 (oversampling)

1. Prior to sampling, a low-pass anti-aliasing filter is

used to attenuate those high frequency
components of the signal that are not essential to
the information being conveyed by the message 𝑓𝑠 = 2𝐵 (Nyquist rate)
signal g(t).
2. The filtered signal is sampled at a rate slightly
higher than the Nyquist rate.
𝑓𝑠 < 2𝐵 (undersampling)

19
 Sampling Theorem – Signal
Reconstruction
𝑓𝑠 > 2𝐵 helps in the design of the
reconstruction filter used to recover the
(a) Anti-alias filtered spectrum of an information-bearing signal.
original signal from its sampled
version.
Design of Reconstruction Filter
1. The reconstruction filter is low-pass
with a passband extending from –W
to W, which is itself determined by
the anti-aliasing filter. (b) Spectrum of instantaneously sampled version of the signal, assuming the
use of a sampling rate greater than the Nyquist rate.
2. The reconstruction filter has a
transition band extending (for
positive frequencies) from W to
(fs – W), where fs is sampling rate.

(c) Magnitude response of reconstruction filter. 20

 Quantization Process
• Transform the continuous sample-amplitude m(nTs) into discrete approximate amplitude v(nTs)
taken from a finite set of possible amplitudes.

m Quantizer v
g( )

v = g(m)

• Quantizer is assumed to be memoryless and instantaneous, which means that the transformation
at time t = nTs is not affected by earlier or later samples of the message signal m(t).

• Such a discrete approximate is adequately good in the sense that any human ear or eye can
detect only finite intensity differences.
 Quantization Process
• Let mk = m(kTs)
• The discrete amplitudes mk, k = 1, 2, …, L, at the quantizer input are called decision levels or
decision thresholds.
• At the quantizer output, the index k is transformed into an amplitude vk that represents all
amplitudes of the cell Jk.
• The discrete amplitudes vk , k = 1, 2,…, L, are called representation levels or reconstruction levels.
• The spacing between two adjacent representation levels is called a quantum or step-size.
• Signal amplitude 𝑚 is specified by the index k if it lies inside the partition cell
𝐽𝑘 : 𝑚𝑘 < 𝑚 ≤ 𝑚𝑘+1 , 𝑘 = 1,2,3, … , 𝐿

Department of Electronics & Communication 22

 Types of Quantizers
• Quantizer (i.e., the device performing the quantization process) is memoryless and instantaneous,
(which means that the transformation at time 𝑡 = 𝑛𝑇𝑠 is not affected by earlier or later samples
of the message signal m(t))

• Types of quantization

– Uniform

• Quantization step sizes are of equal length.

– Non-uniform

• Quantization step sizes are not of equal length.

• Quantizer characteristic can be described by a staircase function
• An alternative classification of quantization
– Midtread - origin lies in the middle of a tread of the staircase like graph
– Midrise - origin lies in the middle of a rising part of the staircase like graph
• Both midtread and midrise are uniform quantizers and are symmetric about the origin.

output output

input input

midtread midrise
 Quantization Noise
• The use of quantization introduces an error defined as the difference between the continuous
input sample m and the quantized output sample v.
• The error is called quantization noise.

Uniform midtread
quantizer in the
previous slide
 Illustration of the quantization process
• Fig illustrates a typical variation of quantization noise as a function of time, assuming the use of
a uniform quantizer of the midtread type.

26
 Quantization Noise
• Define the quantization noise to be Q = M – V = M – g(M), where g( ) is the quantizer.
• Let the message M be uniformly distributed in (–mmax , mmax). So, M has zero mean.
• Assume g( ) is symmetric and of midrise type; then, V = g(M) also has zero-mean, and so does Q
= M – V.
• Then, the step size Δ of the uniform quantizer is given by:
2𝑚max
Δ= Example.
𝐿
where L is the total number of representation levels. mmax =1 D=
1
• Let R denote the number of bits per sample used 2
in the construction of the binary code. L=4
𝐿 = 2𝑅
R = log2(L)
 Quantization Noise
• Let the quantizer input m be the sample value of a zero-mean random variable M.
• Let the quantization error be denoted by the random variable Q of sample value q.
• For uniform quantizer, the sample values of quantization error Q will be bounded by –∆/2 < q ≤ ∆ /2.
• If the step size Δ is sufficiently small (i.e., L is sufficiently large), it is reasonable to assume that the
quantization error (noise) Q is a uniformly distributed random variable.
• The probability density function of the quantization noise as,
1 ∆ ∆
𝑓𝑄 𝑞 = ቐ∆ , − 2 < 𝑞 ≤ 2
0, otherwise
• With the mean of the quantization noise being zero, its variance is the mean-square value,
∆ ∆
2 2 ∆
1 1 𝑞3 2
𝜎𝑄2 =E𝑄 2 2 2
= න 𝑞 𝑓𝑄 𝑞 𝑑𝑞 = න 𝑞 𝑑𝑞 = .
∆ ∆ 3 ∆
−2
∆ ∆
−2 −2

∆2
𝜎𝑄2 = 28
12
 Quantization Noise
∆2
𝜎𝑄2 =
12
2𝑚max ∆2
Substituting Δ = and 𝐿 = 2𝑅 in 𝜎𝑄2 =
𝐿 12
2 2 2
2𝑚max 2𝑚max 4𝑚𝑚𝑎𝑥
𝐿 2𝑅 22𝑅
𝜎𝑄2 = = =
12 12 12

1
𝜎𝑄2 = 𝑚𝑚𝑎𝑥
2
2−2𝑅
3
• Let P denote the average power of the original message signal m(t).
• Then the output signal-to-noise ratio of a uniform quantizer as,
𝑃 3𝑃
𝑆𝑁𝑅 𝑂 = 2= 2 22𝑅
𝜎𝑄 𝑚𝑚𝑎𝑥
• This shows that the output signal-to-noise ratio of a uniform quantizer 𝑆𝑁𝑅 𝑂 increases
exponentially with increasing number of bits per sample, R 29
 Example 1 Sinusoidal Modulating Signal
• Let m(t) = Am cos(2πfmt). Then 𝑃 3𝑃
𝑆𝑁𝑅 𝑂 = 2 = 2 4𝑅
𝜎𝑄 𝑚𝑚𝑎𝑥

Signal-to-(quantization) noise ratio for varying number

of representation levels for sinusoidal modulation
L R SNRO (dB)
32 5 31.8
𝐿 = 2𝑅
64 6 37.8
R = log2(L)
128 7 43.8
256 8 49.8
For sinusoidal modulation, this table provides a basis for making a quick estimate
of the number of bits per sample required for a desired output signal-to-noise ratio.
THANK YOU…