SENSITIVITY

ANALYSIS
Presented by BHARGAV SEERAM, 121202079

DEFINITION
A technique used to determine how different values of an

independent variable will impact a particular dependent

variable under a given set of assumptions. This technique is
used within specific boundaries that will depend on one or
more input variables, such as the effect that changes in
interest rates will have on a bond's price.

Sensitivity analysis is a way to predict the outcome of a

decision if a situation turns out to be different compared to

the key prediction(s).

YOU SHOULD PERFORM A

SENSITIVITY ANALYSIS ANYTIME YOU
Create A Model
Write A Set Of Requirements
Design A System
Make A Decision
Do A Tradeoff Study
Originate A Risk Analysis
Want To Discover The Cost Drivers
3

WHY SENSITIVITY ANALYSIS

Uncertainty in various parameters used in Simulation Models

Feedback loops, Probability Distributions etc.

Values of these parameters cannot be estimated precisely due

to data availability or time constraints.

Less Reliable Models: tested for their sensitivity to the changes

in model components.

Components, to which simulation results are sensitive, need

more attention than other parts of the model.

Parameter sensitivity of the model can be compared with the

information from real system.

HISTORY: THE EARLIEST

SENSITIVITY ANALYSES
The genetics studies on the pea by Gregor Mendel, 1865.
The statistics studies on the Irish hops crops by Gosset

(reported under the pseudonym Student), ca 1890.

SYSTEM DYNAMICS MODEL

The problematic behavior is tried to be explained by system

structure.

The behavior pattern of the system is the main interest of

analysts rather than the specific values of the variables.

Therefore, a behavior-pattern-oriented approach should be

applied to sensitivity analysis

It is difficult to analyze oscillations with correlation based

statistical methods using the values of model variables, hence

this approach is more significant
6

SYSTEM OSCILLATION
System oscillation is the characteristic symptom of negative

feedback structures in which the information used to take

goal-seeking action is delayed

SYSTEM OSCILLATION

SENSITIVITY ANALYSIS IN
LITERATURE
In the study conducted by Ford (1990), sensitivity of results are

measured by partial correlation coefficients which indicate the

strength of linear relationship between two variables after the
effects of other variables are removed

Ford and Flynn (2005) propose Pearson correlation coefficients

instead of the partial ones for simpler sensitivity analysis procedure.

This method is named as screening

Sterman (2000) proposes that numerical sensitivity can be used for

simulation models which work with great numerical precision such

as models in physics or light simulators.

Policy sensitivity is designed as the changes in the optimal policy

when parameters values change (Moxnes, 2005; Sterman, 2000).

9

BEHAVIOR PATTERN
SENSITIVITY
Behavior measures are the subject matter of system dynamics

studies in which the problematic behavior is analyzed with respect to

its structure.

For instance a population in an isolated area, which follows a boom

and bust behavior, can be explained by depleting resources and

vanishing birth rate.

Sensitivity of the behavior to the model parameters can be analyzed

by using peak or equilibrium level of the behavior pattern.

Behavior pattern sensitivity aims to explore the effect of varying

model inputs on the specific behavior measures.

10

BEHAVIOR SENSITIVITY
ANALYSIS METHODOLOGY

SEPARATE
DIFFERENT
BEHAVIOR
MODES
Define

CHOOSE THE
ANALYSIS
PARAMETERS
OF THE
MODEL

DETERMINE
PARAMETER
RANGES AND
DISTRIBUTION

MAKE
SENSITIVITY
SIMULATION
S

EXPLORE
POSSIBLE
BEHAVIORS

Are
there
any
Differen
t
Behavio
r Modes

Behavior
Measure For
Each
Different
Behavior
MAKE
REGRESSION
ANALYSIS
FOR EACH
Calculate
MODE
Behavior
Pattern
Measures for
Each
Behavior

11

SENSITIVITY ANALYSIS
METHODS
Correlation & Screening Method

Ford and Flynn (2005) suggest Pearson correlation in order to conduct

quick sensitivity analysis, called screening.

In this method, correlation coefficients between the output and each
parameter are calculated and plotted against simulation time
Parameters that have high correlation with output variable are concluded
to be the high sensitivity ones

Regression Method
Another convenient method assuming linear relationship between
variables is regression.
In this method, an equation that minimizes the sum of squares of residual
terms is calculated by using ordinary least squares algorithm (Draper and
Smith, 1998).
When a nonlinear relationship is detected at the diagnosis of regression

12

SENSITIVITY ANALYSIS
METHODS (2)
Statistical Analysis (ANOVA) of Clusters in Scatter Plots
Another efficient way of dealing with nonlinearity between system output
and parameters is using statistical analysis of clusters in which output
variable y is plotted against each parameter xj and this plot is subject to
statistical analysis after it is divided into several clusters (Kleijnen and
Helton, 1999a).
This method is a more general way of one-variate sensitivity analysis
since it does not have the linearity assumption between dependent and
independent variables, i.e. this is a statistical model independent
method" (Saltelli et al., 2000).
The scatter plot for each parameter is subject to statistical analysis in
order to detect any non-random pattern.
13

SENSITIVITY ANALYSIS OF A BASIC OSCILLATION SUPPLY CHAIN

MODEL

Figure: Stock Flow Structure of Simple Supply Line Model by Sterman (2000)

14

SENSITIVITY ANALYSIS OF A
BASIC OSCILLATION SUPPLY
CHAIN
MODEL
Supply chains are good examples of material delay formulations that

are rigorously discussed in system dynamics literature.

Supply chains consist of a stock and flow structure for the

acquisition, storage and conversion of inputs into outputs and the

decision rules governing these flows

Include negative feedback loops that create corrective action once

discrepancy arises between the stock and its desired level

The transformation process in each supply chain takes some amount

of time, i.e. there is a time delay in every supply chain structure.

Interaction between negative feedback loops and the time lag may

yield oscillations

15

PATTERN SENSITIVITY ANALYSIS OF

OSCILLATION OF SIMPLE SUPPLY LINE
MODEL
Oscillations are cyclic behavior patterns which are difficult to analyze with

standard statistical techniques, such as screening.

So, sensitivity analysis of oscillatory models should focus on the pattern

measures of these behavior modes

Common measures of an oscillatory pattern are period, first amplitude and

amplitude slope.

Period of an oscillation is amount of time between two successive peaks or

troughs.

This pattern measure indicates how much the system oscillates under

certain circumstances.

One of the critical steps in pattern sensitivity analysis procedure is the

estimation of pattern measures for each simulation run.

16

SENSITIVITY ANALYSIS OF
OSCILLATION PERIOD
In this thesis, periods are estimated by autocorrelation function in BTSII

which is a validation tool for behavior pattern tests of system dynamics

models.

According to the results of regression analysis, stock adjustment time is the

most important parameter of the model.

The second important parameter is acquisition lag which is the average

time lag in supply line. Increasing values of this parameter yields longerperiod oscillatory systems.

17

18

19

REFERENCES
A methodology for statistical sensitivity analysis of system dynamics

models by Mustafa Hekimoglu

([Link]/labs/sesdyn/publications/theses/ms_hekimoglu.Pdf)

20