Decision Analysis and Pharmacoeconomic

Evaluations

By Gizachew T. 1
 3.1. Basic Concepts of Probability

By Gizachew T. 2
Basic concept of Probability
Definition
 The frequentist school of statistics
 The probability of an event as the number of times the event occurs divided by
the number of trials in which it could have occurred ,n, as n approaches infinity.
 P (A) = [The number of times the outcome occures]/[Total umber of trials]

 For example, the probability that a coin will come up heads is 0.5 because,
assuming the coin is fair, as the number of trials (flips of the coin) gets larger
and larger, the observed proportion will be, on average, closer and closer to 0.5.
 Problem: Repetition of events may not be infinite. Some events can happen only
once

By Gizachew T. 3
….Basic concept of Probability…
Definition
 The Bayesian school
 The probability of any event occurring as the personal degree of belief
that the event will occur.
 Therefore, if I personally believe that there is a 70% chance that team A
will win tonight’s game, then that is my probability for this event.
 Problem: statements reflect personal subjective beliefs

By Gizachew T. 4
….Basic concept of Probability…

 Having a range of probabilities depending on the personal beliefs of a

community of clinicians is a useful reflection of reality.
 As more data accumulate, Bayesian and frequentists probabilities tend to
agree, each essentially converging to the mean of the data. When this
occurs, similar inferences will be reached from either viewpoint.

By Gizachew T. 5
 ….Basic concept of Probability…
Concept of Union, Intersection and Complement

 Pink + Gray + yellow = A U B

 Gray = A n B
 Yellow = A’
 Pink = B’

By Gizachew T. slide 6
….Basic concept of Probability…
Rules of probability
 Four basic rules of probability exist. probabilities for events always follow
these four rules.
 [Link] convention, all probabilities are numbers between 0 and 1.
 The value of a probability can only be 0 ≤ p ≤ 1
 A probability of 0 indicates an impossible event, and a probability of 1
indicates an event certain to happen.
 Most events of interest have probabilities that fall between these
extremes.

By Gizachew T. 7
….Basic concept of Probability…
Mutually exclusive events and the additive rule
of probability
 Events are termed disjoint (exclusive) if they have no outcomes in
common.
 If two events are mutually exclusive (disjoint), the probability that one or
the other will occur equals the sum of the probabilities:
 p(A or B) = p(A) + p(B)
 If A and B are two events, not necessarily disjoint, then
 p(A or B) = p(A) + p(B)-p(A and B)

By Gizachew T. 8
….Basic concept of Probability…
Mutually Exclusive events...

 Example
 Role a six sided Die. The possible outcomes (Sampling space) are six
(1,2,3,4,5,6). Each event has equal probability of occurrence (i.e. 1/6).
 Probability of rolling an even number would be:
 p(even) = p(2)+ p(4)+ p(6)
 = (1/6)+(1/6)+(1/6)=1/2

By Gizachew T. 9
….Basic concept of Probability…
Rules…

 [Link] one could list the set of all possible disjoint events of an experiment,
then the probability of one of these events happening is 1.
 The sum of the probabilities that an event will occur and that it will not
occur is equal to 1.

By Gizachew T. 10
4. Independent events and the multiplicative
rule of probability

 For two given events, if the occurrence or nonoccurrence of one doesn’t

affect in any way the occurrence or nonoccurrence of the other, the events
are called independent events (i.e., knowing the outcome of one provides
no information concerning the likelihood that the other will occur)
 Then the probability that both events will occur is given by the product of
their individual probabilities p(A and B) = p (A)p(B)

By Gizachew T. 11
Independent events…
Example
 If the probability of diagnosing a malignant tumor is 0.1, and the
probability that it will rain today is 0.3 (an independent event) then
 The probability of a malignant tumor and rain today is
 0.1 ×0.3 = 0.03, or 3%

By Gizachew T. 12
Pick One
You know the number of black and white balls in each pot but can’t see into the
pot. You have one opportunity to take a ball out. If it is black you win a prize of
$50. Which pot would you choose, the red or blue pot?

5 Black 3 Black
6 White 4 White 6 Black 9 Black
3 White 5 White

By Gizachew T. 13
Pick One!!
5 Black 3 Black 6 Black 9 Black
6 White 4 White 3 White 5 White

5/11 > 3/7 6/9 > 9/14

0.455 > 0.429 0.666 > 0.643
Choose Red
Choose Red

By Gizachew T. 14
WEIRD PROBABILITIES!!
 OK experts! But you get another chance.
 This time the contents from both red pots are
combined and both blue pots combined.
 Which pot would you choose to pick from?

By Gizachew T. 15
WEIRD PROBABILITIES!!
5 Black 3 Black 6 Black 9 Black
6 White 4 White 3 White 5 White

11 Black 12 Black
9 White 9 White

By Gizachew T. 16
WEIRD PROBABILITIES!!

11 Black 12 Black
9 White 9 White

11/20 < 12/21

0.55 < 0.571
Choose Blue!!!
SIMPSON’S PARADOX
By Gizachew T. 17
 Simpson’s paradox
when using two-way tables – tables set up to display the relationship
between two categorical variables – an association or comparison that
holds for all of several groups can disappear or even reverse
direction when the data are combined to form a single group

By Gizachew T. 18
Conditional probability

 What is the probability that a neonate died? Clearly, this depends on a

number of factors, including maternal wellbeing, gestational age, birth
weight, the presence or absence of neonatal illness etc.
 The probability of neonatal death is conditional on other factors and is
not a single constant number by itself.
 Such probabilities are known as conditional probabilities
 Many probability calculations used in health research involve conditional
probabilities

By Gizachew T. 19
Conditional probability…

 If unconditional probabilities can be denoted by Pr( neonatal death), then

conditional probabilities can be denoted by
 Pr(ND| LBW(<2500 gm)), read as “the probability of ND given or
conditional on a LBW baby,” and similarly,
 Pr(ND/ normal birth weight), read as “the probability of ND given a NBW
baby”
 These probabilities are highly relevant in practice and research.
 Conditional profanities follow the rules of probabilities

By Gizachew T. 20
Conditional probability…

 General Notations
 If we denote one event by A, and a second event by B, then
 Pr (A | B), is given by dividing the unconditional probability that these
two events occur together by the unconditional probability that B occurs
 P(A\B) = P(A∩B)/P(B)
 This formula is conveniently rewritten as the following which follows the
Multiplicative Rule.
 P(A∩B) = P(A\B) X P(B)

By Gizachew T. 21
Conditional probability…

 Example : Assume that the overall probability of a patient in the clinic

having endometrial cancer is 10%. Furthermore, suppose that the
sensitivity of endovaginalsonography for diagnosing endometrial cancer is
90%.
 If we let A represent the event that the patient has a positive test result
and let B represent the probability of endometrial cancer in this patient
population
 ( i.e. Pr(B) = 0.1 and Pr(A | B) = 0.9.)
 The probability that a patient in this clinic has both endometrial cancer
and positive results is
 0.1 ×0.9 = 0.09 or 9%

By Gizachew T. 22
Oil Drilling
 A company drills an oil well. The well may be dry (p=0.6), might contain
oil (p=0.1) or contain gas (p=0.3).
 What is the probability of oil, given the result is successful?
 P(oil/successful) = P(oil and successful)/ P
(successful)=P(oil)/P(successful)
= 01/0.4 = 0.25

By Gizachew T. 23
Defective item at a factory
 A company produces an item at either of two plants (A and B). (60% in A
and 40% in B)
 The probability that an item produced at plant A is defective is 0.1, while it
is 0.2 for plant B
 We chose an item from the warehouse and it is defective. What is the
probability that it came from Plant A?

By Gizachew T. 24
Plant A and defective
 P(D/A)= 0.1 , P(D/B)= 0.2
 Calculate P(AD) and P(BD)
 P(D)= 0.1*0.6+0.2*0.4
 P (A|D) = P(AD)/P(D) = 0.06/0.14
= 3/7 =0.429
 P(B|D) = P(BD)/P(D) = 0.08/0.14
= 4/7 = 0.571
By Gizachew T. 25
Bayes' theorem

 It says that you can use conditional probability to make predictions in

reverse.
 Sometimes called the inverse probability law:
 P(B|A) = P(A and B)/P(A) ………………………………1
 P(A|B) = P(A and B)/P(B) ………………………………2
 Solving [1] for P(A and B) and substituting into [2] gives
 Bayes' Theorem: P(A|B) = [P(B|A)][P(A)]/P(B)

By Gizachew T. 26
Bayes' theorem
 P(B\A) =

 P(A\B) =

 P(A\B) =

 P(A\B) =

 P(A\B) =

By Gizachew T. 27
Bayes’ theorem…

 Example 1
 Suppose there is a certain disease randomly found in 0.5% of the general
population. A certain clinical blood test is 99% effective in detecting the
presence of the disease among persons with the disease. But it also yields
false-positive results in 5% of individuals without the disease.
 The following tables show the probabilities that are stipulated in the
example and the probabilities that can be inferred from the stipulated
information:

By Gizachew T. 28
Bayes’ theorem…

 Given
 P(A)= .005 The probability that the disease will be present in any particular
person
 P(~A)= 1—.005 = .995 The probability that the disease will not be present in any
particular person
 P(B|A)= .99 The probability that the test will yield a positive result [B] if the
disease is present [A]
 P(~B|A)= 1—.99 = .01 The probability that the test will yield a negative result [~B]
if the disease is present [A]
 P(B|~A)= .05 The probability that the test will yield a positive result [B] if the
disease is not present [~A]
 P(~B|~A)= 1—.05 = .95 The probability that the test will yield a negative result
[~B] if the disease is not present [~A]

By Gizachew T. 29
Bayes’ theorem…

 Given this information, the derivation of two simple

probabilities is possible using conditional probability
formula
P(B)=[P(B|A)xP(A)]+[P(B|~A)xP( The probability of a
~A)] = [.99 x .005]+[.05 x .995] = positive test result [B],
.0547
irrespective of whether the
disease is present [A] or not
present [~A]

P(~B)=[P(~B|A)xP(A)]+[P(~B|~A)xP(~A)] = The probability of a negative

[.01 x .005]+[.95 x .995] = .9453 test result [~B], irrespective of
whether the disease is present
[A] or not present [~A]
By Gizachew T. 30
Bayes’ theorem…
 Then it is possible to calculate the remaining probabilities.
P(A|B)=[P(B|A)xP(A)]/P(B)= [.99 x The probability that the
.005] / .0547 = .0905 disease is present [A] if the
test result is positive [B]
P(~A|B)=[P(B|~A)xP(~A)]/P(B)= [.05 The probability that the disease
x .995] / .0547 = .09095 is not present [~A] if the test
result is positive [B]
P(~A|~B)=[P(~B|~A)xP(~A)]/P(~B)= The probability that the
[.95 x .995] / .9453 = .99995 disease is absent [~A] if the
test result is negative [~B]
P(A|~B)=[P(~B|A)xP(A)]/P(~B)= [.01 The probability that the
x .005] / .9453 = .00005 disease is present [A] if the
By Gizachew T. 31
test result is negative [~B]
Bayes’ theorem
 Suppose that the background rate of endometrial cancer
seen in patients referred to a particular radiology clinic is
10% and that a diagnostic test is applied that has Pr(A | B)
= 90% sensitivity and Pr(not A | not B) = 80% specificity.
 What is the probability that a patient with positive test
results in fact has endometrial cancer?
 According to Bayes’ theorem, we calculate
 Pr (B | A) =

 0.1 ×0.9/ 0.1 ×0.9 + 0.9 ×0.2 = 0.33 or about 33%.

 In this case, even when a patient has a positive test result,
the chances that the disease is present are less than 50%.

By Gizachew T. 32
Bayes’ theorem

 Similarly, what is the probability that a subject testing negative has

endometrial cancer? Again using Bayes’ theorem,
 Pr (B | not A) =

 0.1 ×0.1/0.1 ×0.1 + 0.9 ×0.8 = 0.013.

 Starting from a background rate of 10% (pretest probability), the
probability of cancer rises to 33% after a positive diagnosis and falls to
approximately 1% after a negative test (post test probabilities).
 Bayes’ theorem allows us to update our probabilities after learning the test
result.

By Gizachew T. 33
Limitations of Randomized Controlled Trials
( RCTs)

 Using RCTs for the basis of pharmacoeconomic assessment has some clear
limitations:
 Efficacy vs. effectiveness.
 Internal vs. external validity
 Timeframe
 Protocol-driven medical care use.
 Sample sizes
 Comparator--Placebo
 Selection of participants
By Gizachew T. 34
…Decision analysis…cont
 It helps in answering questions like
 What is the effectiveness of a new treatment?
 What is the chance of adverse event ?
 What are the costs associated to treatment failure?
 How cost-effective are treatment alternatives ?
 What statistical distribution surrounds costs probabilities or outcomes?
 What impact might non-adherence have on cost effectiveness?
 Based on my threshold to pay for health care is this cost effective option?

By Gizachew T. slide 35
…Decision analysis…cont
 Long history within public and private sectors
 Used in healthcare technology assessment to inform decision making at
both individual and population levels
 At its essence, a ‘simulated, mathematical, disease state model’
 May utilize expert opinion, literature-based results, data from prospective
or retrospective analyses

By Gizachew T. slide 36
…Decision analysis…cont
 Provides the conceptual foundation for cost-
effectiveness, cost-utility, and cost-benefit analyses
 Game theory: ‘one-person game

Option • Option 1
1

Option • Option 2
2

Option • Option 3
3

By Gizachew T. slide 37
…Decision analysis…cont
Key Attributes

 All relevant aspects of the decision  Alternative course of

and relevant strategies are explicitly action
articulated:  Novel treatments
 Base Case  Standard of care
 Perspective  Watchful waiting
 Time Horizon  Chance events
 Treatment strategies  Probability of
treatment failure
 Probabilities
 Probability of adverse
 Costs reaction
 Outcomes  Consequences
 Uncertainty  Economic, Clinical or
Humanistic outcomes

By Gizachew T. slide 38
 …Decision analysis…cont
When to use?

 When randomized clinical trials (RCTs)…

 Do not sufficiently capture data needed to support pharmacoeconomic
decision making
 Measure only surrogate outcomes
 Compare treatments only to placebo (or less than gold standard) When
analysts want…
 To predict routine clinical care (effectiveness) from RCT data (efficacy)
 To examine institution-specific results
 To identify optimal strategies based on value

By Gizachew T. slide 39
 …Decision analysis…cont
advantages and limitations

 Advantages  Limitations
 Timely  Potentially complicated structural
 Inexpensive and content validity aspects
 Ethical  “Black-box” perception
 May compare treatment  Potential for bias with discretionary
options without additional nature of methods and data
RCTs selection
 Ability to synthesize  Method of combining data or
current state of knowledge synthesizing the current state of
knowledge?
 Reliability of estimates
 Results are only as robust as underlying
model structure and data permit
 Often requires assumptions to be made

By Gizachew T. slide 40
Typical Steps in Conducting a Decision Analysis
 1) Establish the research question
 2) Define the perspective (i.e., ‘Costs to whom’)
 3) Define the base case(s) of analysis (e.g., patient and disease
 characteristics)
 4) Specify treatment modalities and choose appropriate
comparator(s)
 (e.g., least costly, standard of care, most-commonly used)
 5) Model the disease state, define the time horizon, and choose
 surrogate and/or final outcomes
 6) Populate the model with probabilities, costs, and outcome data
 7) Verify the model, calculate and report results
 8) Conduct additional sensitivity analyses

By Gizachew T. slide 41
Types of Modeling Methods in DA
Types of modeling methods frequently used in health technology assessment:
 Decision trees
 Markov
 Cohort
 Monte Carlo
 Microsimulation
 Fixed-time advance
 Discrete-event, Time-to-event (without and with queuing for resources)
 Agent-based

By Gizachew T. 42
1. Use of models
 What is a ‘model’?
 A simplification of reality to capture the ‘essence’ of the
problem with the minimum level of complexity
 Why use a model?
 To synthesise data from multiple sources
 To handle uncertainty & assumptions, e.g.
 To extrapolate from intermediate to final outcomes

 To predict outcomes that are unknown or are unethical to collect

 Handle ambiguity of clinical data and variations in interpretation

By Gizachew T. 43
Types of models
 Descriptive
 describes
 Prescriptive
 suggests
 Deterministic
 certainty
 Stochastic (decision tree and Markov modelling )
 probability

By Gizachew T. 44
Example deterministic, prescriptive model
Become ill

See a doctor Do not see a doctor

Obtain a prescription

Take medication

Recover rapidly Slow recovery

By Gizachew T. 45
Example stochastic, prescriptive model
Rapid (decision-tree)
recovery
Leaves
(p)
See doctor
Decision node
1-(p)
Slow recovery

Become ill

Rapid recovery
(q)
Branches
Do not see doctor
1-(q)
Slow recovery
Chance node
By Gizachew T. 46
Decision Trees
 A decision tree is a diagrammatic representation of the possible outcomes and
events used in decision analysis
 The questions to be asked in an analysis of a question are arranged as a series of
decision or chance nodes, each node with resulting branches, creating a tree
effect
 The sequential steps proceed with each step depending on the decision or
probability outcome from the preceding step1

By Gizachew T. 47
Decision Tree Example
Simple tree fragment modeling complications of anticoagulant therapy 1

By Gizachew T. 48
Decision Tree Components
Simple tree decision trees embody the essential paradigm of decision analysis.
Specifically, all decisions may be decomposed into three broadly-defined
components:
1. Decision node – point in time when a choice is made among competing
strategies( RECTANGULAR)
2. Decision strategy – set of actions or events consequent to a decision(CIRCLE)
3. Outcome nodes – terminal branches of tree that represent outcomes of a
strategy.1 Multiple outcomes (payoffs) may be assigned. (TRIANGULAR)

By Gizachew T. 49
Calculating Expected Value of a Decision Tree
 The expected value of a decision tree is calculated by
“averaging out” or “folding back the branches of the tree
 The value(s) of each strategy is path probability to the
terminal node multiplied by the payoff(s) at the terminal
node
 Branching probabilities can be deterministic represented by
point values (e.g., 0.6) or stochastic represented by
probability distributions (e.g., normal, exponential)
 Uncertainty around branching probabilities and terminal
node values is examined with sensitivity analysis1

By Gizachew T.
50
Advantages of Decision Trees
 Graphical – can diagrammatically represent decision alternatives, chance events, and
possible outcomes; visual approach assists with comprehending decision sequences
and dependencies
 Efficient - can quickly express complex alternatives clearly, and easily modify as new
information becomes available
 Complementary – can use in conjunction with other methodologies, e.g., append
recursive methods to terminal nodes1

By Gizachew T. 51
Disadvantages of Decision Trees
 Must assume population being examined can be modeled in the
aggregate; if being applied to an individual, assumption is made that
aggregate probabilities are relevant to the individual1
 Does not specify when events occur
 Assumes that each event can occur only once

By Gizachew T.
52
Steps in Decision Analysis

 1. Identify and bound the problem

 2. Structure the problem
 3. Gather data
 4. Analyze the tree
 5. Perform sensitivity analysis

By Gizachew T. 53
Example: SARS

 • SARS is a new highly lethal respiratory infection

 – 30% of patients with SARS will die if untreated
 • Drug A has been developed for SARS
 – Significant serious adverse drug effects have been reported
 • Those having an ADE have a high risk of death

By Gizachew T. 54
Step 1: Identify and bound
the problem

 • What is the decision problem;

 – what is the clinical question?
 • What are the potential alternative actions?
 • What events follow the decision?

By Gizachew T. 55
….DA decision tree
 Step 1: Identify and bound the problem
 What is the decision problem?
 Should we treat with Drug A?
 What are the potential alternative actions?
 Treat with Drug A
 No Treatment
 events follow the decision?
 ADEs
 Survival
 Death

By Gizachew T. 56
….DA decision tree
 Step 2: Structure the problem
 • Use a decision tree
 • A decision tree depicts graphically the components of the decision
 problems and relates actions to consequences

By Gizachew T. 57
….DA decision tree

By Gizachew T. 58
….DA decision tree
 Step 3: Gather data
 Conduct systematic search where appropriate
 Can use RCTs, meta‐analysis, expert opinion, etc.
 Use best estimate for “base‐case” analysis
 Use 95% CI’s or ranges for sensitivity analysis

By Gizachew T. 59
….DA decision tree
 Drug A Data
 Risk of ADE = 0.10
 Risk of death if ADE = 0.40
 “Cure” rate if no ADE = 0.90

By Gizachew T. 60
….DA decision tree

By Gizachew T. 61
….DA decision tree
 Step 4: Analyzing the tree
 Calculate expected value of each strategy
 Also referred to as “rolling back” or taking the average of the tree
 Start at terminal node and multiply probabilities as you trace tree to origin to
get probability of outcome
 Sum weighted outcomes for each strategy

By Gizachew T. 62
….DA decision tree

By Gizachew T. 63
….DA decision tree
 Step 5) Run sensitivity analyses
 Perform 1‐way sensitivity analyses on all parameters to debug tree
 Vary probabilities from 0 to 1; response of model to changes should be logical
 Set all costs/outcomes to zero; strategies should have same expected value

By Gizachew T. 64
….DA decision tree

By Gizachew T. 65
…SA
 One-way or univariate SA,
 Two-way or bivariate SA,
 Multivariate SA,
 Best-case analysis
 Worst case analysis
 Probabilistic SA, also referred to as Monte Carlo analysis,

By Gizachew T. 66
 DA… Example 2

 Example: Boy with abdominal pain 12 yo boy with 8 hrs of abdominal pain
and nausea, who vomited once. He ate at a restaurant earlier in the day.
There is no significant past history, no meds.
 Exam: scared boy with diffuse abdominal pain, but only mild guarding in
the periumbilical area. CBC is only mildly elevated. He is being considered
for admission for “R/O Appendicitis”

By Gizachew T. 67
 Decision elements

 Decision Elements
 Choices are to admit and observe for the next 6 hrs, or take to the OR now.
 Probability data (surgeon guestimates):
 appendicitis: 50%
 rupture of observed for 6 hrs: 20%
 operative mortality with immediate surgery : 1%
 operative mortality after rupture: 4%
 operative mortality with stabilization: 0.02%
 surgical morbidity 5 time greater after rupture
 Morbidity=1 day lost

By Gizachew T. 68
Structuring the tree

By Gizachew T. 69
Assigning probability values

By Gizachew T. 70
Calculating average effect

By Gizachew T. 71
SA…One way

By Gizachew T. 72
SA… two way

By Gizachew T. 73
SA…Three way

By Gizachew T. 74
 Decision tree model can be used to calculate cost effectiveness of
competing alternatives and show the best alternative with its ICER

By Gizachew T. 75
 Steps in this ‘deterministic’ example
 To obtain the
 Structure of the decision tree
 Path probabilities
 Path costs
 Average costs of Drug A and Drug B
 Average’ effectiveness of Drug A and Drug B
 Average cost-effectiveness of Drug A and Drug B
 Incremental cost-effectiveness of the Drug B versus Drug

By Gizachew T. slide 76
 Average Cost = ∑(path cost*path probability)
 Average effect= ∑(path probability*Path utility)
 CER= Average cost/Average effect
 ICER= (difference of Av cost)/(difference of Av effect)

By Gizachew T. 77
Exercise 1

By Gizachew T. 78
Exercise 2

By Gizachew T. 79
 Exercise 3

 You are asked to select the cost effective drug to be included in a district
formulary from three drugs designated as drug A, B and C The following
table shows the costs and probabilities associated with each drug. Do a
decision analysis using decision tree and propose which drug will be cost
effective; which drug will be the second cost effective? Use the data in the
following table

By Gizachew T. 80
Data for exercise 3

Parameters Drug ‘’A’’ Drug ‘’B’’ Drug ‘’C’’

Cure rate 70% 90% 80%
Significant adverse 20% 10% 15%
event rate
Cost per course of $ 100 $ 150 $120
treatment
Cost to treat adverse $ 25 $ 25 $20
event
Cost of treatment $ 200 $ 200 $150
failure

By Gizachew T. 81
By Gizachew T. 82
 Markov Process
Simple Example

Weather:
• raining today 40% rain tomorrow
60% no rain tomorrow

• not raining today 20% rain tomorrow

80% no rain tomorrow
Stochastic FSM:
0.4 0.6
0.8

rain no rain

0.2
By Gizachew T. 83
 Markov Process
Simple Example

Weather:
• raining today 40% rain tomorrow
60% no rain tomorrow

• not raining today 20% rain tomorrow

80% no rain tomorrow

The transition matrix:

• Stochastic matrix:
 0.4 0.6  Rows sum up to 1
P    • Double stochastic matrix:
 0.2 0.8  Rows and columns sum up to 1

By Gizachew T. 84
Markov Process
Coke vs. Pepsi Example
• Given that a person’s last cola purchase was Coke, there is a 90% chance that
his next cola purchase will also be Coke.
• If a person’s last cola purchase was Pepsi, there is an 80% chance that his next
cola purchase will also be Pepsi.

transition matrix: 0.1

0.9 0.8
0.9 0.1
P 
coke pepsi

 0.2 0.8 0.2

By Gizachew T. 85
Markov Process
Coke vs. Pepsi Example (cont)

Given that a person is currently a Pepsi purchaser, what is the probability that
he will purchase Coke two purchases from now?
Pr[ Pepsi?Coke ] =
Pr[ PepsiCokeCoke ] + Pr[ Pepsi Pepsi Coke ] =
0.2 * 0.9 + 0.8 * 0.2 = 0.34

00.9.9 00.1.1 0.9 0.1 0.83 0.17 

P 
2
    
00.2.2 00.8.8 0.2 0.8 0.34 0.66
Pepsi  ? ?  Coke
By Gizachew T. 86
 Markov Process
Coke vs. Pepsi Example (cont)

Given that a person is currently a Coke purchaser, what is the probability that
he will purchase Pepsi three purchases from now?

0.9 0.1 0.83 0.17   0.781 0.219

P 
3
    
 0.2 0.8 0.34 0.66   0.438 0.562 

By Gizachew T. 87
Markov Process
Coke vs. Pepsi Example (cont)
•Assume each person makes one cola purchase per week
•Suppose 60% of all people now drink Coke, and 40% drink Pepsi
•What fraction of people will be drinking Coke three weeks from now?

0.9 0.1  0.781 0.219

P  P 
3

 0.2 0.8  0.438 0.562 
Pr[X3=Coke] = 0.6 * 0.781 + 0.4 * 0.438 = 0.6438

Qi - the distribution in week i

Q0=(0.6,0.4) - initial distribution
Q3= Q0 * P3 =(0.6438,0.3562)

By Gizachew T. 88
Markov Process
Coke vs. Pepsi Example (cont)
Simulation:

2/3

0.9 0.1 2
2 3 1 
3   3
1
3 
 0 . 2 0 . 8 
Pr[Xi = Coke]

stationary distribution

0.9 0.1 0.8

coke pepsi

0.2

week - i
By Gizachew T. 89
Markov model

 A Markov process is simply a mathematical representation of the health

states in which a patient might find him- or herself and the likelihood of
transitioning between those states.
 • The Markov process itself, when it is evaluated, calculates the average life
expectancy of a cohort proceeding through the Markov process.

By Gizachew T. 90
Markov model
 Use Markov Model when
 transitions into and out of health states are possible, e.g.,recurrent events
 modeling a complex disease
 probabilities vary over time
 the timeframe of the analysis is lengthy
 a decision tree would become too complex

By Gizachew T. 91
Introduction . . .

Danny’s ear infections

 if well this week, 80% chance well
next week
 if infected this week, 50% chance
infected next week
 probabilities remain constant from
week to week

By Gizachew T. 92
 Introduction . . .

Danny’s ear infections

 Suppose well this week. What is:
 p (well next week)?
 p (well in 2 weeks)?
 p (well in 10 weeks)?

 Long-run averages:
 how many infections annually?
 length of infected episode?
 % time infected?

 How might prophylaxis alter results?

By Gizachew T. 93
Introduction . . . Assessment of prognosis
A critical factor in decision making

 Decisions must be made over long time

horizons
 Risks are continuous over time
 Important events may happen more than once
 Estimates are required of:
 Life expectancy
 Event-free survival
 Frequency/length of hospitalization
 Sensitivity to alternative interventions

By Gizachew T. 94
Methods . . .
Markov models
Central idea

 states of health
 Finite
 Mutually exclusive
 Collectively exhaustive
 Capture disease characteristics
(prognosis, cost, quality of life, etc.)
 Transitions
 Probabilistic
 One each cycle

By Gizachew T. 95
 Principal Elements of a
Markov Models

Mutually exclusive health states

 Transition probabilities
 Cycle length
 Utility value and/or cost per health
state

By Gizachew T. 96
 Markovian Assumption
 Probability of transition depends only on current
health state and not on past health states
(memoryless property)
 Health state definitions should include all relevant
history
» history of precancerous lesion
» history of acute myocardial infarction
» time since HIV seroconversion

By Gizachew T. 97
Building a Markov Model
 Determine health states
 Determine transitions
 Choose cycle length
 Estimate transition probabilities
 Estimate state utilities and costs per cycle
 Calculate
 Sensitivity analysis

By Gizachew T. 98
Methods . . .

Health States

Well Sick

By Gizachew T. 99
Building a Markov Model
 Determine health states
 Determine transitions
 Choose cycle length
 Estimate transition probabilities
 Estimate state utilities and costs per cycle
 Calculate
 Sensitivity analysis

By Gizachew T. 100
Methods . . .

State-transition diagram:

W S

By Gizachew T. 101
State Transition Diagram

WELL SICK

DEAD

By Gizachew T. 102
 Modeling devices:
Methods . . .

Initial, transient, and absorbing states

W S R

By Gizachew T. 103
Methods . . . Modeling devices:
Tunnel states

Pre- Post Post Post

Op
op Op 1 Op 2 Op 3

By Gizachew T. 104
Building a Markov Model
 Determine health states
 Determine transitions
 Choose cycle length
 Estimate transition probabilities
 Estimate state utilities and costs per cycle
 Calculate
 Sensitivity analysis

By Gizachew T. 105
 Cycle
A brief time interval during which patients
within a cohort may make a transition into
another health state or remain in the current
health state.

By Gizachew T. 106
Building a Markov Model
 Determine health states
 Determine transitions
 Choose cycle length
 Estimate transition probabilities
 Estimate state utilities and costs per cycle
 Calculate
 Sensitivity analysis

By Gizachew T. 107
 Transition Probability
The chance that patients in a particular health
state will transfer to another health state during
the course of a cycle.

By Gizachew T. 108
Methods . . .

.2

.5
.8
W S

.5

By Gizachew T. 109
State Transition Diagram
Pij = transition prob from state i to state j
P 11

P22
P12
WELL SICK

P13 P23

DEAD
1. WELL
2. SICK
P33 3. DEAD
By Gizachew T. 110
 Transition probability matrix:
Methods . . .

Another useful tool

NEXT PERIOD
P1
Well Sick

Well
THIS
PERIOD

Sick

By Gizachew T. 111
Methods . . . Transition probability matrix:
Another useful tool

NEXT PERIOD
P1
Well Sick

Well .8 .2
THIS
PERIOD

.5 .5
Sick

By Gizachew T. 112
Methods . . . Suppose well this period.
What is p(well in 2 periods)?

PERIOD AFTER NEXT

P2
Well Sick

Well
THIS
PERIOD

Sick

By Gizachew T. 113
Methods . . . Suppose well this period.
What is p(well in 2 periods)?

PERIOD AFTER NEXT

P2
Well Sick

Well .74 .26

THIS
PERIOD
.65 .35
Sick

By Gizachew T. 114
 Matrix multiplication
Methods . . .

A shorthand

 Can accommodate any number of

states
 Can be carried out efficiently on a
computer
 Can produce “n-step” transition
probability matrices

By Gizachew T. 115
Long-run behavior . . .

In the long-run . . .
 movement between states persists

 but the influence of initial states wears off

 transition probabilities represent proportion of

time spent in a state

 long-run = equilibrium = steady state

By Gizachew T. 116
Effect of intervention . . .

Back to Danny: Suppose drugs reduce

incidence of infection by 50%

NEXT PERIOD
P1
Well Sick

Well
THIS
PERIOD

Sick

By Gizachew T. 117
Effect of intervention . . .

Back to Danny: Suppose drugs reduce

incidence of infection by 50%

NEXT PERIOD
P1
Well Sick

Well .9 .1
THIS
PERIOD
.5 .5
Sick

By Gizachew T. 118
Effect of intervention . . .

Suppose better drugs speed recovery by 50%

NEXT PERIOD
P1
Well Sick

Well
THIS
PERIOD

Sick

By Gizachew T. 119
Effect of intervention . . .

Suppose better drugs speed recovery by 50%

NEXT PERIOD
P1
Well Sick

Well .8 .2
THIS
PERIOD
.75 .25
Sick

By Gizachew T. 120
Estimating prognosis . . .

Estimating prognosis

0.3 0.5 0.4

W S

0.2 0.6
D

1.0
1. Test your intuition: What
are the steady state
transition probabilities?
2. Average life expectancy?
3. Average symptom-free
survival?
By Gizachew T. 121
Building a Markov Model
 Determine health states
 Determine transitions
 Choose cycle length
 Estimate transition probabilities
 Estimate state utilities and costs per cycle
 Calculate
 Sensitivity analysis

By Gizachew T. 122
Health State “Utility”
 The value of occupying a particular
health state for one cycle.
» A value of 1 will calculate life expectancy
» A value of 1/(1+r)cycle will calculate
discounted life expectancy
» A utility value (between 0 and 1) will
calculate quality-adjusted life expectancy
» A cost value will calculate lifetime costs
By Gizachew T. 123
Quality-of-Life Adjustments

1.0 0.5 0.0

WELL SICK DEAD

By Gizachew T. 124
 Building a Markov Model
 Determine health states
 Determine transitions
 Choose cycle length
 Estimate transition probabilities
 Estimate state utilities and costs per cycle
 Calculate
 Sensitivity analysis

By Gizachew T. 125
 Common Methods of Evaluation

• Cohort simulation
• Hypothetical cohort of patients
transition through the model
simultaneously
• Monte Carlo simulation
• First order simulation randomly selects
a patient from the hypothetical cohort
and they transition through the model
one at a time

By Gizachew T. 126
Estimating prognosis . . .

Cohort simulation
 Hypothetical population cohort is
assigned to initial states

 Transition probabilities re-distribute

the population at each cycle

 Can compute average patient-cycles for

each state

By Gizachew T. 127
Estimating prognosis . . .

Monte Carlo simulation

 Considers hypothetical population one
individual at a time

 At each cycle, a random number generator

simulates transitions

 Repeating this process for large enough sample

produces point estimates and variances

By Gizachew T. 128
 Monte Carlo Simulation

Well Sick Dead

Well Sick Dead

Well Sick Dead

WWSD
Well Sick Dead
By Gizachew T. 129
 Cohort Simulation can
Estimate Survival Curves
Time

t WELL SICK DEAD

t+1 WELL SICK DEAD

By Gizachew T. 130
Quality-of-Life Adjustments

1.0 0.5 0.0

WELL SICK DEAD

By Gizachew T. 131
 Transition Probability Matrix:

WELL SICK DEAD

WELL 0.75 0.20 0.05
SICK 0 0.70 0.30
DEAD 0 0 1

Suppose everyone starts in the WELL state

By Gizachew T. 132
 Cohort Simulation
Cycle Total
Cycle Well Sick Dead QA life QALE
0 1 0 0 0 0
1 0.75 0.20 0.05 0.85 0.85
2 0.56 0.29 0.15 0.71 1.56
3 0.42 0.32 0.26 0.58 2.14
4 0.32 0.31 0.38 0.47 2.61

N 0 0 1 0 4.33

By Gizachew T. 133
 Markov Approximation to LE

Survival Probability 1
0.8 Life Expectancy = Area Under the Survival Curve

0.6
0.4
0.2
...
0
0 5 10 15 20
Time (years)
By Gizachew T. 134
 Benefits of a Model in CEA

Incorporates the benefits and costs beyond time

horizon of existing data

Considers all relevant clinical strategies

Incorporate data from multiple sources

Evaluates “what if” scenarios

By Gizachew T. 135
 Extend Beyond Time Horizon

• Can “translate” an intermediate endpoint

(e.g., increase in CD4 count, decrease in
blood pressure) into clinical endpoints
such as life years saved or quality-
adjusted life years gained
• Incorporates important long-term effects
of treatment

By Gizachew T. 136
 Consider All Relevant
Strategies
• Comparison of interventions often not
compared head-to-head in a trial
• pharmaceutical vs. educational interventions
• Clinical trials do not always consider those
strategies which would be most relevant in a
cost-effectiveness analysis
• Too many plausible strategies to be feasible
for a clinical trial

By Gizachew T. 137
 Data from Multiple Sources

• Clinical trial data from primary

sources
• Existing databases
• Studies reported in the medical
literature
• Expert opinion

By Gizachew T. 138
 “What If” Scenarios

 How great would the diagnostic accuracy of a

noninvasive imaging test for coronary arteries have to
be to replace the tests used in current practice?
 How effective does a treatment need to be, and at what
duration of treatment effect, to replace current
therapy?
 Identifies important gaps in our knowledge

By Gizachew T. 139
 From the Panel on Cost-effectiveness
in Health and Medicine:
 Evidence for effectiveness may be obtained from
RCTs, observational data, uncontrolled
experiments, descriptive series, and expert opinion.
 Where direct primary or secondary empirical
evaluation of effectiveness is not possible (for
example, in important subpopulations or in
differing time frames), the use of modeling to
estimate effectiveness is a valid mode of scientific
inquiry.

By Gizachew T. 140