ST3236/MA3238: Tutorial 11
Due date: August 31st, Online
Wanjie Wang
1. (Bayes Formula)
Consider the weather with three possible states: sunny, cloudy, and rainy. Given the weather
in last hour, the weather in the next hour has the following conditional probabilities:
P (Sunny → Sunny) = a11
P (Sunny → Cloudy) = a12
P (Sunny → Rainy) = a13
P (Cloudy → Sunny) = a21
P (Cloudy → Cloudy) = a22
P (Cloudy → Rainy) = a23
P (Rainy → Sunny) = a31
P (Rainy → Cloudy) = a32
P (Rainy → Rainy) = a33
Without any information, the probability of weather is that P (Sunny) = p1 , P (Cloudy) = p2 ,
and P (Rainy) = p3 .
(a) Prove that ai1 + ai2 + ai3 = 1 for any i = 1, 2, 3.
(b) A traveller arrives and finds the current weather as Sunny. With this additional infor-
mation, what is the conditional probability that in last hour, it was raining?
(c) Following (b), could you identify the conditional distribution of the weather in last hour?
1
All rights reserved by NUS. Reproduction or distribution of lecture notes/tutorials/quiz/exam without the written
permission of the sponsor is prohibited.
1
(d) Suppose a11 = a22 = a33 = 0.6, a12 = a13 = 0.2, a21 = a31 = 0.1, a23 = a32 = 0.3. The
probability follows that p1 = 0.2, p2 = 0.4, p3 = 0.4. Find the conditional distribution
in part (c).
2. (Markovian Property) For the following examples, decide whether they are Markov chain and
justify.
(a) The price of a particular stock is determined by its previous two days’ prices. If the
stock was higher on both of the previous two days, there’s a 60% chance it will go up
today. If it was lower on both previous days, there’s a 70% chance it will go down today.
Otherwise, the stock has an equal chance of going up or down. Consider the process
formed by the stock price.
(b) In a board game, a player rolls a die and moves forward by the number of spaces rolled.
The probability of rolling any given number on the die does not depend on the players
current position on the board. Consider the process formed by the location of this player.
(c) A user browsing the web moves from one page to another. The probability that the user
will click on a link and move to another page depends only on the current page they are
on and not on any pages they visited before reaching the current page. Consider the
process formed by the webpages this user has browsed.
(d) A particle starts at position 0 on a number line. At each time step, it moves one unit
to the right with probability 0.5 and one unit to the left with probability 0.5. The
movement only depends on the current position. Consider the process formed by this
particle.
(e) The probability that a person becomes infected by a disease depends on the number of
infected individuals in the past three days. If there were more than two new infections
on any of the last three days, the probability of getting infected today is high. If there
were fewer, the probability is low.
Consider the process recording the number of infected individuals.
2
3. A Markov chain X0 , X1 , · · · , has the transition probability matrix
0 1 2
0 0.6 0.3 0.1
P=
1 0.3 0.3 0.4
2 0.4 0.1 0.5
Given that the process starts in state X0 = 1,
(a) Find P(2) .
(b) Find the probability P (X0 = 1, X1 = 0, X2 = 2).
(c) Find the probability P (X2 = 2).
(d) Find the probability P (X1 = 0|X2 = 2).
4. A Markov chain X0 , X1 , · · · on state 0, 1, 2 has the transition probability matrix
0 1 2
0 0.1 0.2 0.7
P= .
1 0.9 0.1 0
2 0.1 0.8 0.1
Suppose the initial distribution is p0 = P (X0 = 0) = 0.3, p1 = P (X0 = 1) = 0.4, and
p2 = P (X0 = 2) = 0.3.
(a) Draw the diagram with transition probability.
(b) Determine the distribution for X1 .
(c) Determine the distribution for X2 .
(d) Find P (X0 = 0, X1 = 1, X2 = 2). Is it the same with P (X0 = 0)P (X1 = 1)P (X2 = 2)?
Why or why not?
3
Practice Problems
Notes:
• This section is for students who want to practice more
• NO submission required.
• NO solution. NO tutorials about it. Some problems have final answers but NOT ALL.
• You are encouraged to discuss with your colleagues.
1. (choice question)
A miner is trapped in a mine containing 3 doors. The first door leads to a tunnel that will
take him to safety after 3 hours of travel. The second door leads to a tunnel that will return
him to the mine after 5 hours of travel. The third door leads to a tunnel that will return him
to the mine after 7 hours of travel. We assume that the miner is at all times equally likely to
choose any one of the doors. Let Xn denote the door this miner chooses at the n-th time he
gets back.
(a) What is the state space of Xn ?
(b) Is {Xn , n = 1, 2, · · ·} a Markov Chain? If yes, is it stationary? Briefly explain.
(c) Since Door 1 will lead to the safety, so we set P (Xn+1 = 1|Xn = 1) = 1, which means
he stays at that state forever. Please find out the transition probability matrix for this
process.
(d) Please draw the diagram of this process.
(e) Note that X1 has equal probability on {1, 2, 3}. Find the distribution of X2 .
(f) Find the expected travel time of this miner during the first choice.
(g) Given X1 = 1, then it takes 0 hours of travel for X2 . Given that X1 is equally likely to
be {1, 2, 3}, find the expected travel time of this miner during the second choice,