Our Interest and Goals
• Study tools to characterize uncertainty.
- Probability theory, random events, random variables
• Study related tools to characterize non-deterministic signals.
- Random processes, statistics
• Analyze systems with non-deterministic inputs and outputs.
- Linear systems with random inputs
- Communication channels with noise
- Communication networks with uncertain delays
M Tummala & C W Therrien 2012 1
Typical Electrical Engineering Problems
A noisy binary communication channel
• The channel can be twisted pair, coaxial cable, fiber optic cable,
or wireless medium.
• The channel introduces noise and thereby bit errors.
h(n)
x(n) y(n)
Station A Channel Station B
00110001010... 00100001010...
M Tummala & C W Therrien 2012 2
Typical Electrical Engineering Problems (cont’d)
Signal detection
• Desired target signal is buried in noise.
signal signal noise
• Determine the presence or absence of the desired signal.
• Filter the signal out of noise.
• Demodulate the signal.
M Tummala & C W Therrien 2012 3
Typical Electrical Engineering Problems (cont’d)
Networks
• In large computer networks, there are limited resources
(e.g., bandwidth, routers, switches, printers and other devices)
that need to be shared by the users.
– User jobs/packets are queued and assigned service based
on predefined criteria.
– Demand is uncertain and service time is also uncertain.
– Delay from the time the service is requested to the time it is
completed is uncertain.
• Similar considerations exist for telephone networks, multiuser
computer networks, and other communication networks.
M Tummala & C W Therrien 2012 4
Basics of Probability
Probability
Assignment
Outcome Events and
Random Sample
Experiment Space Their Probabilities
Define Events
• In a random experiment, the outcome is uncertain.
• The entire collection of outcomes is the sample space S.
• An event consists of a single or a group of outcomes.
• Probability Pr[A] is a measure of likelihood of the event A.
–
M Tummala & C W Therrien 2012 5
Example Illustrating Terms
Roll a die once. All faces are equally likely.
• Sample space [discrete]:
S = {1,2,3,4,5,6}
• Define events:
A1 = {Odd numbered face} = {1,3,5}
A2 = {Face value < 3} = {1,2}
• Probability assignment:
Pr[A1] = 3/6 = 1/2
Pr[A2] = 2/6 = 1/3
M Tummala & C W Therrien 2012 6
Example: (cont’d)
Roll a die once. All faces are equally likely.
• Sample space [discrete]:
S = {1,2,3,4,5,6}
• Define events:
A1 = {Odd numbered face} = {1,3,5}
A2 = {Face value < 3} = {1,2}
A3 = {Face value > 2} = ?
• Probability assignment:
Pr[A1] = 3/6 = 1/2
Pr[A2] = 2/6 = 1/3
M Tummala & C W Therrien 2012 7
Example: (cont’d)
Roll a die once. All faces are equally likely.
• Sample space [discrete]:
S = {1,2,3,4,5,6}
• Define events:
A1 = {Odd numbered face} = {1,3,5}
A2 = {Face value < 3} = {1,2}
A3 = {Face value > 2} = {3,4,5,6}
• Probability assignment:
Pr[A1] = 3/6 = 1/2
Pr[A2] = 2/6 = 1/3
Pr[A3] = 4/6 = 2/3
M Tummala & C W Therrien 2012 8
Random Variables and Random Proceses
Mapping X (s) = x
s real number line
x
Sample space
X (s, t)
Space of time
functions
M Tummala & C W Therrien 2012 9
Outcome Mapping
(1,2) 3
(2,3) 5
(4,1) 5
M Tummala & C W Therrien 2012 10
Example of Random Process: Speech Waveforms – “hello”
M Tummala & C W Therrien 2012 11