Utility theory

Sunil Simon
Elements of a decision problem/game
Actions (A) – alternatives from which an agent/player can choose.

Outcomes (X) – possible consequences that can result from the actions of each
player.

Preferences (⪰) – Ranking of the set of possible outcomes.

Preference relation
Weak preference relation: x ⪰ y – x is at least as good as y.

Strict preference relation: x is strictly better than y.

x ≻ y iff x ⪰ y and y ⪰
/x

Indifference relation: x and y are equally good.

x ≈ y iff x ⪰ y and y ⪰ x

Sunil Simon Utility theory

Properties of the preference ordering

1 Completeness: for all outcomes x and y either x ⪰ y or y ⪰ x.

2 Reflexivity: for every outcome x ∈ X, x ⪰ x.

3 Transitivity: for all outcomes x, y and z, if x ⪰ y and y ⪰ z then x ⪰ z.

Rational preference relation – a preference relation which is complete, reflexive and

transitive.

Sunil Simon Utility theory

Utility function

A utility function (or payoff function) u ∶ X → R represents the preference relation ⪰

if for any pair x, y ∈ X, u(x) ≥ u(y) iff x ⪰ y

Proposition. If the set of outcomes X is finite then any rational preference relation
over X can be represented by a utility function.

Proof. Since the preference relation is complete and transitive, find a least preferred
outcome x1 ∈ X such that all other outcomes y ∈ X, y ⪰ x1 .

X1 = {x ∈ X ∣ x ≈ x1 } – worst outcome equivalence set.

Choose the least preferred outcome x2 ∈ X ∖ X1 . Let X2 = {x ∈ X ∖ X1 ∣ x ≈ x2 }.

Continue until the best outcome equivalence set Xn is created. Since X is finite and ⪰
is rational such a finite collection of equivalence sets exists.

Choose n arbitrary values un > un−1 > . . . > u1 and let u(x) = uk for x ∈ Xk .

Sunil Simon Utility theory

Utility function

Proposition. Let ⪰ be a complete, reflexive and transitive preference relation over X.

Suppose that u is a utility function representing ⪰, then for every monotonically
strictly increasing function v ∶ R → R, the composition v ○ u defined by

(v ○ u)(x) = v(u(x))

is also a utility function representing ⪰.

Sunil Simon Utility theory

Preference relation over uncertain outcomes

Lottery
Lotteries over X is L = {[p1 (x1 ), . . . , pk (xk )] ∣ pi ≥ 0 and p1 + p2 + . . . + pk = 1}.

Example

L1 = [ 12 (x1 ), 12 (x2 ), 0(x3 )].

L2 = [0(x1 ), 13 (x2 ), 23 (x3 )].
x1 = [1(x1 ), 0(x2 ), 0(x3 )].

Observation. The set of outcomes X can be regarded as a subset of the set of lotteries
L.

Sunil Simon Utility theory

Utility

Utility function
A utility function u represents the preference relation ⪰ if for any pair of lotteries L, L′
we have u(L) ≥ u(L′ ) iff L ⪰ L′ .

Linear utility function, von Neumann - Morgenstern utility function

A utility function u is said to be linear if for every lottery L = [p1 (x1 ), . . . , pk (xk )] the
following holds:
u(L) = p1 u(x1 ) + p2 u(x2 ) + . . . + pk u(xk ).

Question: Which preference relation of a player can be represented by a linear utility

function?

Sunil Simon Utility theory

The axioms of utility theory

Compound lottery
L̂ = [q1 (L1 ), q2 (L2 ), . . . , qm (Lm )].

Example: [L̂ = 43 (L1 ), 14 (L2 )] where

L1 = [ 23 (x1 ), 13 (x2 )] and L2 = [ 12 (x3 ), 21 (x4 )].

Utility function
A utility function u ∶ L̂ → R represents the preference relation ⪰ if for any pair of
compound lotteries L̂, L̂′ we have u(L̂) ≥ u(L̂′ ) iff L̂ ⪰ L̂′ .

Sunil Simon Utility theory

The axioms of utility theory
Axiom 1 – Continuity. For every triplet of outcomes x ⪰ y ⪰ z, there exists a θ ∈ [0, 1]
such that y ≈ [θ(x), (1 − θ)(z)].

Axiom 2 – Monotonicity. Let α, β ∈ [0, 1] and suppose that x ≻ y then

[α(x), (1 − α)(y)] ⪰ [β(x), (1 − β)(y)] iff α ≥ β.

Lemma. If a preference relation satisfies the Axioms of Continuity and Monotonicity,

and if x ⪰ y ⪰ z and x ≻ z, then the value of θ defined in the Axiom of Continuity is
unique.
Corollary. If a preference relation ⪰ satisfies continuity and monotonicity and if
xk ≻ x1 , then for each j = 1, . . . , k there exists a unique θj such that

xj ≈ [θj (xk ), (1 − θj )(x1 )]

Sunil Simon Utility theory

The axioms of utility theory

Axiom 3 – Simplification. Let L̂ = [q1 (L1 ), q2 (L2 ), . . . , qm (Lm )] and for each
j j j
j ∶ 1 ≤ j ≤ m let Lj = [p1 (x1 ), p2 (x2 ), . . . , pk (xk )].

For each i = 1, . . . , k, let ri = q1 p1i + q2 p2i + . . . + qm pm

i .

Consider the simple lottery L = [r1 (x1 ), r2 (x2 ), . . . , rk (xk )], then L̂ ≈ L.

Example
L = [ 12 (x1 ), 14 (x2 ), 18 (x5 ), 18 (x7 )]

L1 = [ 23 (x1 ), 13 (x2 )]
L2 = [ 12 (x5 ), 12 (x7 )]

L̂ = [ 34 (L1 ), 14 (L2 )]

Sunil Simon Utility theory

The axioms of utility theory

Axiom 4 – Independence. Let L̂ = [q1 (L1 ), q2 (L2 ), . . . , qm (Lm )] and M be a simple

lottery. If Lj ≈ M then

L̂ ≈ [q1 (L1 ), . . . qj−1 (Lj−1 ), qj (M), qj+1 (Lj+1 ), . . . , qm (Lm )].

Sunil Simon Utility theory

A characterization theorem

Theorem. If the preference relation ⪰ over L̂ is complete, transitive and satisfies the
four von Neumann-Morgenstern axioms then ⪰ can be represented by a linear utility
function.

Sunil Simon Utility theory

A characterization theorem
Proof. Assume that the most desired outcome xk ≻ x1 . By Lemma 1, for each
outcome xj we have xj ≈ [θj (xk ), (1 − θj )(x1 )].
Utility function. Suppose L̂ = [q1 (L1 ), q2 (L2 ), . . . , qm (Lm )] and for each
j j j
j ∶ 1 ≤ j ≤ m let Lj = [p1 (x1 ), p2 (x2 ), . . . , pk (xk )].
Define for each i = 1, . . . , k, ri = q1 p1i + q2 p2i + . . . + qm pm
i .

u(L̂) = r1 θ1 + r2 θ2 + . . . + rk θk .

For every simple lottery L = [p1 (x1 ), . . . , pk (xk )], u(L) = ∑kj=1 pj θj
Outcome xj is same as the lottery L = [1(xj )] which is same as L̂ = [1(L)]. So
outcome of L̂ is xj with probability 1. So we have
⎧
⎪
⎪1 if i = j,
ri = ⎨
⎪
⎩0 if i ≠ j.
⎪

We deduce that u(xj ) = θj .

Sunil Simon Utility theory
A characterization theorem

To show: The function u is linear.

Need to show that for each simple lottery L = [p1 (x1 ), . . . , pk (xk )],

k
u(L) = ∑ pj u(xj )
j=1

We have,
1 u(L) = ∑kj=1 pj θj ,
2 u(xj ) = θj .

(1) implies that u(L) = ∑kj=1 pj θj and (2) implies that ∑kj=1 pj u(xj ) = ∑kj=1 pj θj .

Sunil Simon Utility theory

A characterization theorem

To show: The function u is a utility function.

Need to show that for any pair of compound lotteries L̂ and L̂′ ,

L̂ ⪰ L̂′ iff u(L̂) ≥ u(L̂′ ).

Claim 2. L̂ ≈ [u(L̂)(xk ), (1 − u(L̂))(x1 )] for every L̂.

Monotonicity. Suppose x ≻ y, [α(x), (1 − α)(y)] ⪰ [β(x), (1 − β)(y)] iff α ≥ β .

Assuming Claim 2, the result follows from monotonicity of ⪰.

Sunil Simon Utility theory

A characterization theorem
Claim 2. L̂ ≈ [u(L̂)(xk ), (1 − u(L̂)(x1 )] for every L̂.

Proof. Let L̂ = [q1 (L1 ), q2 (L2 ), . . . , qm (Lm )] and for each j ∶ 1 ≤ j ≤ m let
j j j
Lj = [p1 (x1 ), p2 (x2 ), . . . , pk (xk )].
Let ri = q1 pi + q2 pi + . . . + qm pm
1 2
i (the probability that the outcome is xi in L̂).

By simplification axiom, L̂ ≈ [r1 (x1 ), . . . , rk (xk )].

Let Mi = [θi (xk ), (1 − θi )(x1 )] for every 1 ≤ i ≤ k. By definition xi ≈ Mi . Thus k

application of the independence axiom

L̂ ≈ [r1 (M1 ), . . . , rk (Mk )].

Let r∗ be the total probability of xk in the lottery on the RHS.

k
r∗ = ∑ ri θi = u(L̂).
i=1

By simplification axiom, L̂ ≈ [r∗ (xk ), (1 − r∗ )(x1 )] = [u(L̂)(xk ), (1 − u(L̂))(x1 )].

Sunil Simon Utility theory