Chapter 8: Markov

Models
Course: Reliability and Maintenance Theory
In this Chapter 8, we study the various states of a repairable system. A multicomponent, repairable
system will have a number of possible states, depending on how many of its components are in operation.
The state of the system at time t is denoted 𝑋(𝑡), and we are interested in finding the probability that
the system is in a specific state at time t . We also find the steady-state probabilities, or the average
proportion of time the system is in the various states. The presentation will be limited to a special class
of stochastic processes 𝑋 𝑡 , 𝑡 ≥ 0 having the Markov property. Such a stochastic process is called a
Markov process and is characterized by its lack of memory. If a Markov process is in state j at time t ,
we will get no more knowledge about its future states by knowing the history of the process up to time t .
Introduction
Introduction Contd.
State Space Method For finding Out system
Reliability/ Availability
A system is described by its state and by the possible transition between the
states. The various system states are defined by the states of the system
components comprising the system. By the state space method the system is
not restricted to having only two possible states. The components may have a
number of different states such as functioning, derated, standby, completely
failed or under maintenance. The various failure modes may also be defined as
states. The transition between the states are caused by various mechanism and
activity such as failures, replacements, repairs and switching operations.
 Cold Standby System of Order 2
Consider a system of two components each with constant failure rate 𝜆 and
repair rate 𝜇.
It is assumed that both components are originally operable. One component is
actually in operation and the other is in inactive standby. The component in
operation operates until it fails at which time the standby component begins
operation and repairs are done on the failed component. When the failed
component has been repaired, it becomes an inactive standby. If a failure
occurs, them there is no inactive standby them both components are in failed
state and the system has failed.
It is assumed that switching is perfect and no warmup is required.
 Contd.
Let 𝑖 denotes the state of the system at time 𝑡; 𝑖 = 0,1,2.
State 0: Both components are operable but only one is operating.
State 1: One component has failed and the other one is operating.
State 2: Both components have failed.
Let 𝑝! (𝑡) be the probability hat the system is in state 1 at time 𝑡 since the states are exhaustive and mutually exclusive.
$
, 𝑝! (𝑡) = 1
!"#
𝐹 𝑡 = 1 − 𝑝$ 𝑡 = 𝑝# 𝑡 + 𝑝%(𝑡)
Since it is assumed that the failure rate of the component is constant 𝜆 therefore, the failure time distribution is
exponential with mean time failure 1/𝜆. If 𝑋 is the lifetime of the component then
𝑃 𝑋 ≤ 𝑥 = 1 − 𝑒 &'(
Probability system is working fine till time x
CDF of exponential

It is further assumed that the repair rate of the component is constant 𝜇 therefore, the repair time distribution is
exponential with mean time to repair 1/µ. If 𝑌 is the repair time of the component then
𝑃 𝑌 > 𝑦 = 𝑒 &')
 Contd.
1. Probability that a component survives till 𝑡 + ∆𝑡 given that the component has survived
till 𝑡.
𝑃[𝑋 > 𝑡 + ∆𝑡, 𝑋 > 𝑡]
𝑃 𝑋 > 𝑡 + ∆𝑡 𝑋 > 𝑡 = = 𝑒 !"∆$
𝑃[𝑋 > 𝑡]
("∆$)* ("∆$)+
= 1 − 𝜆∆𝑡 + + +⋯
'! )!
= 1 − 𝜆∆𝑡 + 𝑂(∆𝑡)
2. Similarly,
𝑃[𝑌 > 𝑡 + ∆𝑡, 𝑌 > 𝑡]
𝑃 𝑌 > 𝑡 + ∆𝑡 𝑌 > 𝑡 = = 𝑒 !*∆$
𝑃[𝑌 > 𝑡]
= 1 − 𝜇∆𝑡 + 𝑂(∆𝑡)
 Contd.
3. The probability of more than one change of states in an interval of length
∆𝑡 is negligible. i.e.,
Pr 𝐹𝑎𝑖𝑙𝑢𝑟𝑒 . Pr repair = (1 − 𝑒 ()∆+ )(1 − 𝑒 (,∆+ )
= 𝜆∆𝑡 + 𝑂(∆𝑡) 𝜇∆𝑡 + 𝑂(∆𝑡) = 𝑂 ∆𝑡
Differential Difference Equations
Contd.
Contd.
 Boundary
Conditions &
Laplace
Transform
Contd.
Laplace Inverse & Reliability Measures
 Other Models to Cover
• Parallel System of order 2
• A one unit system
• Series system with repair
• n unit cold standby system with r parallel repairs
• Standby System with repair
• (m/n) system with r parallel repairs