EECE 442 Communication Systems Oct.
12, 2024
American University of Beirut, Fall 2024–2025 Handout #10
Due: Saturday, Oct. 19, 2024
Problem Set 5
Reminder on Collaboration Policy
The following is an acceptable form of collaboration: discuss with your classmates possible
approaches to solving the problems, and then have each one fill in the details and hand-
write her/his own solutions independently.
An unacceptable form of dealing with homework is to copy a solution that someone else
has written.
At the top of each homework you turn in, list all sources of information you used, apart
of course from the text, books on reserve for this course or discussions with the Prof.
A brief note such as “did problem 7 with May Berite in study group” or “used this AI
engine to get some directions” would be sufficient.
In general, we expect students to adhere to basic, common sense concepts of academic
honesty. Presenting another’s work as if it were your own, or cheating in exams will not
be tolerated.
Problem 5.1
Consider a continuous random variable U with a Probability Density Function (PDF)
pU (u) parametrized by two positive scalars a, b > 0 as shown below:
pU (u)
✻
✲
−b b u
(a) What are the values of parameters a and b that will guarantee that pU (.) is a valid
PDF?
(b) Using the optimality conditions, find the optimal 3-points quantizer of U .
(c) What is the resulting Mean Squared Error (MSE) distortion of you quantizer in
(b)?
This study source was downloaded by 100000887184767 from [Link] on 11-02-2025 12:20:36 GMT -06:00
[Link]
Problem 5.2
Consider a continuous random variable U that has the following probability density func-
tion:
1
pU (u) = e−|u| .
2
(a) Apply for three iterations the Lloyd-Max algorithm to find an optimal 3-points
quantizer.
(b) Use the necessary conditions of optimality to find an optimal 3-points quantizer of
U.
(HINT: one of the 3 points will be zero.)
(c) What is the resulting mean squared error of the quantizer obtained in (b)?
Problem 5.3
Part I:
An analog sequence source produces every second a continuous random variable U , with
a Probability Density Function (PDF) of the form
fU (u) = ae−λ|u|
where a is a positive scalar.
We would like to design a scalar quantizer for this source.
(a) For what value of a is fU (.) a valid PDF?
(b) Using the “Mean-Squared Error” distortion measure, solve the necessary conditions
of optimality to find an optimal two-points quantizer for the source.
Part II:
The objective of this part is to derive necessary conditions of optimality for a different
distortion measure, namely the “Mean Absolute Error (MAE)” distortion.
More precisely, you would like to use the distortion measure:
d(u, v) = |u − v|,
as opposed to the square of the difference. The quality measure of a quantizer is naturally
the average distortion and is equal to:
E [d(U, V (U ))] = E U − V (U ) .
This study source was downloaded by 100000887184767 from [Link] on 11-02-2025 12:20:36 GMT -06:00
[Link]
(c) Prove the following statement:
Z
arg min fU (u) u − α du = median of U,
α
where the median of U is defined to be a value “m” such that the probability of U
being less than or equal to m is equal to the probability that U is larger than m
(and hence, both probabilities are equal to half.)
(d) For a quantizer with given M representation points, what are the optimal quanti-
zation regions?
(e) For a quantizer with given M quantization regions, what are the optimal represen-
tation points?
Part III:
Now consider the analog sequence source of part I.
(f) Using the MAE distortion measure, solve the necessary conditions of optimality to
find an optimal two-points quantizer.
(g) Is the optimal quantizer in Part I “better” than that of Part III?
Problem 5.4
Consider a discrete source X that produces every second a value in the set
X = {1, 2, 4, 5, 8, 9},
with Probability Mass Function (PMF)
{0.3, 0.3, 0.2, 0.1, 0.05, 0.05}
respectively.
Part I
(a) Find an optimal symbol-by-symbol source code for the source X.
(b) What is the average length of your codebook in (a) (in bits per second)?
(c) Without computing the entropy of the source, provide a solid argument for why
your data compressor in (a) is not ideal (i.e., it does not compress up to entropy.)
This study source was downloaded by 100000887184767 from [Link] on 11-02-2025 12:20:36 GMT -06:00
[Link]
Part II
(d) Find a Shannon code for the source X.
(e) What is the average length of your codebook in (d) (in bits per second)?
Part III
When computing the entropy of the source you find that it is equal to 2.2710 bits/second.
Assume that your system requirements are to generate no more than 2 bits per second
at the output of the source encoder.
(f) Propose a source encoder that will “perform relatively well” while meeting the
requirements. You are to specify the characteristics of all the components of your
encoder.
Hint:
You may find the following values useful:
log2 (0.03125) = −5, log2 (0.0625) = −4, log2 (0.125) = −3,
log2 (0.25) = −2, log2 (0.5) = −1.
Problem 5.5
The random variable U has a Probability Density Function (PDF):
1
pU (u) = a · , u ∈ R.
1 + π 2 u2
where a is a positive constant.
(a) Find the constant a for pU (u) to be a valid PDF.
(b) You decide to use the “Mean-Squared Error” (MSE) as a distortion measure.
Find an optimal two-points symbol-by-symbol quantizer.
(c) Your friend, who is well-versed in probability theory, is not surprised by the prob-
lems you faced in (b). She argues that the MSE is not a good distortion measure
here. She suggests using the distortion measure:
d(x, x̂) = |x − x̂|.
Solve the necessary conditions of optimality to find an optimal 2 points quantizer.
Note: You may find the following identities useful:
d 1
arctan(x) = .
dx 1 + x2
This study source was downloaded by 100000887184767 from [Link] on 11-02-2025 12:20:36 GMT -06:00
[Link]
Powered by TCPDF ([Link])