ECE 361: Digital Communications

Lecture 1: Discrete Nature of information

Introduction
The currency of today’s information age is digital: bits. Digital communication is reliable
transmission of this currency over an unreliable physical medium. It is an interesting ques-
tion to ask why very different types of information sources such as voice and video are
represented using a common currency, but we will not pursue this question in this course. It
is quite a loaded question and the final word on the matter is not yet said; answers to this
question in certain communication contexts are provided in a separate field of study known
as information theory. The block diagram in Figure 1 shows a high level representation of a
typical communication system. The discrete message source continuously outputs a stream
of bits that represent the information we would like to transmit. Bits are abstract entities
that need to be mapped to a physical quantity, such as an electromagnetic signal, to be
transmitted over a physical medium.

Distortion

?
Message Source - Transmitter - Channel - Receiver

Figure 1: The basic block diagram of a communication system

The behavior of the physical medium is uncertain: what you get is not a deterministic
function of what you send; this uncertainty is the essence of communication. While the
behavior of the channel1 over one experiment cannot be predicted, the average behavior,
averaged over many experiments turns out to be well behaved in many physically interesting
scenarios. The characterization of the average behavior, or in other words, the statistical
characterization of the physical medium is crucial to understanding how to communicate
the bits reliably to the receiver. A primary component of the a communication engineer’s
tool-box is robust and reasonable statistical models of important physical channels such as
the wireline telephone channel and the wireless channels.

A Simple Noise Model

We will begin with a simple form of a physical medium where we only transmit and receive
voltages (real numbers). The received voltage y, is the transmitted voltage x, plus “noise”
w:
y =x+w (1)
1
Channel is a term we will use throughout these notes to denote the unreliable physical medium.
The simplest model of the noise is that w is strictly within a certain range, say ±σth . In
other words, we receive a voltage that is within ±σth Volts from the voltage we transmitted.

A Simple Communication Scheme

Suppose we want to send a single bit across this channel. We can do this by transmitting a
voltage v0 to transmit an information content of the bit being “zero”, and a voltage v1 when
transmitting an information content of the bit being “one”. As long as

|v0 − v1 | > 2σth , (2)

we can be certain that our communication of the one bit of information is reliable over this
channel. Physically, the voltage transmitted corresponds to some energy being spent: we
can say that the energy spent in transmitting a voltage v Volts is (proportional to) v 2 Joules.
In this context, a natural question to ask is the following: how many bits can we reliable
communicate with an energy constraint of E Joules?
Some thought lets us come up with the following transmission scheme: we choose to
transmit one of a collection of discrete voltage levels:
n √ √ √ √ o
− E, − E + 2σth , . . . , − E + 2kσth , . . . , + E , (3)
√
where we have assumed for simplicity that E is divisible by σth . So, we can communicate
one of √
E
1+ (4)
σth
discrete voltage levels entirely reliably to the receiver. This corresponds to
à √ !
E
log2 1 + (5)
σth

bits being reliable communicated to the receiver (why?). The diagram in Figure 2 demon-
strates one possible mapping
√ between the 4 sequences of 2 bits to the 4 discrete voltage levels
being transmitted. Here E = 3σth and
√
vk = − E + 2kσth , k = 0, 1, 2, 3. (6)

00
r 01
r 10
r 11
r -
v0 v1 v2 v3

Figure 2: Mapping from bits to voltage levels.

Relation between Energy and Reliable Information Transmitted
For a given energy constraint E, the number of bits we can communicate reliably is, from
(5), Ã √ !
E
log2 1 + . (7)
σth
A natural sort of question that the communication engineer is interested in is the following:
if we want to send an additional bit reliably how much more energy do we need to expend?
We can use the above expression to answer this question: the new energy energy Ẽ required
to send an extra bit of informamtion reliably has to satisfy:
 √  Ã √ !
Ẽ E
log2 1 +  = 1 + log 1 +
2 , (8)
σth σth
√ Ã √ !
Ẽ E
1+ = 2 1+ , (9)
σth σth
q √
Ẽ = σth + 2 E. (10)

In other words, we need to more than quadruple the energy constraint to send just one extra
bit of information reliably.
Another interesting thing to note is that the amount of reliable communication trans-
mitted depends on the ratio between the transmit energy budget E and the energy of the
2 2
noise σth . This ratio, E/σth , is called the signal to noise ratio and will feature prominently
in the other additive noise models we will see.

Looking Forward
This simple example of a channel model gave us a feel for simple transmission and reception
strategies. It also gave us an idea of how a physical resource such as energy is related to
the amount of information we can communicate reliably. The deterministic channel model
we have used here is rather simplistic; in particular, the choice of σth might have to be
overly conservative if we have ensure that the additive noise has to lie in the range ±σth
with full certainty. If we are willing to tolerate some error in reliable communication, we
can set a lower range σth in our channel model and thus allowing for a higher rate of reliable
communication of information. This is the topic of the next lecture.