Using Indicators to Measure Progress

and Performance
Dominique Senn, seecon gmbh

Using Indicators to Measure Progress and Performance 1

Find this presentation and more on [Link]
[Link]

Copyright & Disclaimer

Copy it, adapt it, use it – but acknowledge the source!
Copyright
Included in the SSWM Toolbox are materials from various organisations and sources. Those materials are open source. Following the open-
source concept for capacity building and non-profit use, copying and adapting is allowed provided proper acknowledgement of the source
is made (see below). The publication of these materials in the SSWM Toolbox does not alter any existing copyrights. Material published in
the SSWM Toolbox for the first time follows the same open-source concept, with all rights remaining with the original authors or producing
organisations.

To view an official copy of the the Creative Commons Attribution Works 3.0 Unported License we build upon, visit
[Link] This agreement officially states that:
You are free to:
• Share - to copy, distribute and transmit this document
• Remix - to adapt this document. We would appreciate receiving a copy of any changes that you have made to improve this
document.
Under the following conditions:
• Attribution: You must always give the original authors or publishing agencies credit for the document or picture you are using.

Disclaimer
The contents of the SSWM Toolbox reflect the opinions of the respective authors and not necessarily the official opinion of the funding or
supporting partner organisations.
Depending on the initial situations and respective local circumstances, there is no guarantee that single measures described in the toolbox
will make the local water and sanitation system more sustainable. The main aim of the SSWM Toolbox is to be a reference tool to provide
ideas for improving the local water and sanitation situation in a sustainable manner. Results depend largely on the respective situation
and the implementation and combination of the measures described. An in-depth analysis of respective advantages and disadvantages and
the suitability of the measure is necessary in every single case. We do not assume any responsibility for and make no warranty with
respect to the results that may be obtained from the use of the information provided.

Using Indicators to Measure Progress and Performance

Find this presentation and more on [Link]
[Link]

Contents

1. Introduction
2. What Are Indicators
3. The Selection of Indicators
4. Why Use Indicators
5. What to Use Indicators for
6. The Development of Indicators
7. Typical Indicators in SSWM
8. Advantages and Disadvantages
9. References

Using Indicators to Measure Progress and Performance 3

Find this presentation and more on [Link]
[Link]

1. Introduction

Every project and programme has specific objectives. In order to assess

the progress and performance of the project (e.g. to analyse whether
targets are met, to give account of the progress to donor agencies or to
show accountability to the public, funders, donors and stakeholder
groups), monitoring and evaluations need to be carried out.

Indicators
provide an
effective tool for
measuring
progress and
performance.
Source: [Link] [Accessed: 23.10.2013]

Using Indicators to Measure Progress and Performance 4

Find this presentation and more on [Link]
[Link]

2. What Are Indicators

Definitions

• OECD/DAC: “A quantitative or qualitative factor or variable that

provides a simple and reliable means to measure achievement, to
reflect changes connected to an intervention, or to help assess the
performance of a development actor.”

• USAID: “A variable, which purpose it is to measure change in a

phenomena or process.”

• UKAID & United States Institute of Peace: An Indicator “refers to a

measure used to demonstrate change in a situation, or the progress
in, or results of, an activity, project, or program.”

• European Commission: “A description of the project’s objectives in

terms of quantity, quality, target group(s), time and place.”
Source: MDF 2005, UKAID & UNITED STATES INSTITUTE FO R PEACE n.y.

Using Indicators to Measure Progress and Performance 5

Find this presentation and more on [Link]
[Link]

2. What Are Indicators

Example

Specific indicators have been developed in order to measure progress

towards the Millennium Development Goals.

Goals and Targets Indicators

Goal 1: Eradicate extreme poverty and hunger

Target 1.A: Halve, between 1990 and 2015, the proportion of people whose 1. Proportion of population below $1 (PPP) per day
income is less than one dollar a day 2. Poverty gap ratio
3. Share of poorest quintile in national consumption
Target 1.B: Achieve full and productive employment and decent work for all, 1. Growth rate of GDP per person employed
including women and young people 2. Employment-to-population ratio
3. Proportion of employed people living below $1 (PPP) per day
4. Proportion of own-account and contributing family workers in total
employment
Target 1.C: Halve, between 1990 and 2015, the proportion of 1. Prevalence of underweight children under-five years of age
people who suffer from hunger 2. Proportion of population below minimum level of dietary energy
consumption
Goal 2: Achieve universal primary education
Target 2.A: Ensure that, by 2015, children everywhere, boys and girls alike, will 1. Net enrolment ratio in primary education
be able to complete a full course of primary schooling 2. Proportion of pupils starting grade 1 who reach last grade of primary
3. Literacy rate of 15-24 year-olds, women and men

For the full list, see: [Link]

Using Indicators to Measure Progress and Performance 6

Find this presentation and more on [Link]
[Link]

3. The Selection of Indicators

(1/2)

Generally an indicator focuses on a small, manageable set of

information that gives a sense of the bigger picture.
→The choice of indicators is important as to whether it gives sufficient
“sense of the bigger picture”.
→ Only well selected indicators can assist in maintaining a focus on the
important work areas and take strategic decisions to address problem
areas.
→The careful selection of indicators makes a project sustainable and
allows responsible persons to act.

Using Indicators to Measure Progress and Performance 7

Find this presentation and more on [Link]
[Link]

3. The Selection of Indicators

(2/2)

The term “minimum indicators” is used in recognition that:

• It is better to start with a small set of indicators that are feasible
to monitor and to improve over time.
• There are many other indicators that could be used to measure
progress beyond this basic level.

Using Indicators to Measure Progress and Performance 8

Find this presentation and more on [Link]
[Link]

4. Why Use Indicators

An indicator is used to answer the question:
How do we know if and how much we have achieved our objective?

Indicators are useful to:

• Measure progress and performance over time against various water
and sanitation objectives to evaluate the effect of policy actions and
plans.
• Provide information on the status, trend or performance of a given
system to inform the project team (responsible organisation),
stakeholders, the public and policy makers in a simplified way.
• Translate data into policy relevant information (describe, show
trends, communicate results of implementing objectives).
• Identify areas for increased attention by an organisation.

Using Indicators to Measure Progress and Performance 9

Find this presentation and more on [Link]
[Link]

6. The Development of Indicators

Criteria for Developing Indicators

Good indicators should meet certain criteria. Three different concepts

are presented on the following slides: SMART indicators, SPICED
indicators and CREAM indicators.
While SMART describes the properties of the indicators themselves,
SPICED relates more to how indicators should be used.

But keep in mind: the most important point in developing indicators is:
Don’t be too ambitious! → Start with what can be realistically done
or else failure is guaranteed.

Using Indicators to Measure Progress and Performance 10

Find this presentation and more on [Link]
[Link]

6. The Development of Indicators

SMART Indicators
The indicators need to be clearly articulated, well defined
S SPECIFIC and focused. They should be clear to people with a basic
knowledge of the issue, programme or initiative.
The indicator should be measurable, that is, it has the
capacity to be counted, observed, analysed, tested, or

M MEASURABLE
challenged. The indicators should be able to determine the
degree of completion or attainment. Using the same
methodology and information, findings should be
repeatable and comparable.

A ATTAINABLE
The target attached to the indicator should be achievable
within the scopes of the project/ programme.
The indicators should be able to detect change and be

R RELEVANT
related to the specific situation they are “indicating”
information about. They should be measurable at an
appropriate scale.

T TIME-BOUND
The indicator should be attached to a time frame. Term
dates for measurement should be included.
Source: Save the Ch ild ren , [Link] me/Home/smart-indicators [Accessed: 17.12.2013]

Using Indicators to Measure Progress and Performance 11

Find this presentation and more on [Link]
[Link]

6. The Development of Indicators

SPICED Indicators
Informants have a special position or experience that gives them unique

S SUBJECTIVE
insights which may yield a very high return on the investigators time. In
this sense, what may be seen by others as 'anecdotal' becomes critical
data because of the source’s value.
Indicators should be developed together with those best placed to
P PARTICIPATORY assess them. This means involving a project's ultimate beneficiaries, but
it can also mean involving local staff and other stakeholders.

I INTERPRETED &
COMMUNICABLE
Locally defined indicators may not mean much to other stakeholders, so
they often need to be explained.

The validity of assessment needs to be cross-checked, by comparing

C CROSS-CHECKED &
COMPARED
different indicators and progress, and by using different informants,
methods, and researchers.
The process of setting and assessing indicators should be empowering in
E EMPOWERING itself and allow groups and individuals to reflect critically on their
changing situation.
There should be a deliberate effort to seek out different indicators from

D DIVERSE &
DISAGGREGATED
a range of groups, especially men and women. This information needs to
be recorded in such a way that these differences can be assessed over
time.
Source: MDF 2005

Using Indicators to Measure Progress and Performance 12

Find this presentation and more on [Link]
[Link]

6. The Development of Indicators

CREAM Indicators

Indicators should be...

C CLEAR ...Precise and unambiguous

R RELEVANT ...Appropriate to the subject at hand

E ECONOMIC ...Available at reasonable cost

A ADEQUATE ...Able to provide sufficient basis to assess performance

M MONITORABLE ...Amenable to independent validation

Source: UKAID & UNITED STATES INSTITUTE FOR PEACE n.y.

Using Indicators to Measure Progress and Performance 13

Find this presentation and more on [Link]
[Link]

7. Typical Indicators in SSWM

(Adapted from MUGA and MIHELCIC 2008)

• Functional indicators (e.g. adaptability, durability and reliability of

the system)
• Economic indicators (e.g. capital costs, operation and maintenance
costs and user costs)
• Environmental indicators covering resource use (e.g. nutrient
reuse) and emissions (e.g. emissions of carbon dioxide)
• Social-cultural indicators (e.g. public participation, acceptance,
and stimulation of sustainable behaviour)

Using Indicators to Measure Progress and Performance 14

Find this presentation and more on [Link]
[Link]

7. Typical Indicators in SSWM

Examples of Indicators for Water Sources Management
• Water withdrawal: absolute or per-person value of yearly water
withdrawal gives a measure of the importance of water in the
country’s economy.
• Ground water level development (rise or fall of groundwater levels,
pollution).
• Water sources quality (e.g. turbidity, minerals content, pathogens,
chemical quality).

Water quality assessment with different indicators (biodiversity, chemical

substances, nutrients, toxics, sediments, etc.).
Source: [Link] ortcard in [Link] [Accessed: 10.12.2013]

Using Indicators to Measure Progress and Performance 15

Find this presentation and more on [Link]
[Link]

7. Typical Indicators in SSWM

Examples of Indicators for Water Purification

• Treatment efficiency indicator (e.g. removal of indicator bacteria)

Removal efficiency of organic

nitrogen in different
treatment methods (1-
nitrification, 2-oxidation pond,
3-chemical coagulation, 4-
chlorination, 5-ammonia
removal, 6-filtration, 7-
reverse osmosis).

Source: [Link] [Link]/books/water-

treatmen t/relationship-of-algae-to-water-pollution-and-
waste-water-treatmen t [Accessed: 10.12.2013]

Using Indicators to Measure Progress and Performance 16

Find this presentation and more on [Link]
[Link]

7. Typical Indicators in SSWM

Examples of Indicators for Water Use

• Percentage or number of people not served with improved drinking

water and extension of piped water supply.

Number of household
connections per 100 residents
in urban sub-Saharan Africa.
Source: WSP & WUP 2003

• Domestic water use efficiency (per capita use of water).

• Agricultural water use efficiency and efficacy (water demand per
irrigated area).
• Industrial water use.
• Awareness and actual use of water-saving appliances.
Using Indicators to Measure Progress and Performance 17
Find this presentation and more on [Link]
[Link]

7. Typical Indicators in SSWM

Examples of Indicators for Wastewater Collection

• Percentage of wastewater collected to total wastewater

production.

Particularly in peri-urban and

rural areas in developing
countries, wastewater is rarely
collected, representing high
health risks to the people
living in these aras.
Source: [Link]
[Link]

Using Indicators to Measure Progress and Performance 18

Find this presentation and more on [Link]
[Link]

7. Typical Indicators in SSWM

Examples of Indicators for Wastewater Treatment
• Percentage or number of people not served with basic sanitation.

Percentage of households
having a sewerage connection
in sub-Saharan African cities.
Source: WSP & WUP 2003

• Number of people affected by water borne diseases (e.g.

diarrhoea, cholera).
• Investment in drinking water supply and sanitation.
• Removal of pollutants.
• Awareness and actual use of safe sanitation systems.
Using Indicators to Measure Progress and Performance 19
Find this presentation and more on [Link]
[Link]

7. Typical Indicators in SSWM

Examples of Indicators for Reuse and Recharge of Nutrients and Water

• Percentage of reclaimed water used in agriculture to total

agricultural water use.
• Percentage of nutrients used from recycled sources.

Proportion of recycled water

to withdrawn water.

Source:
[Link]
i/environment/#EN8 [Accessed: 10.12.2013]

Using Indicators to Measure Progress and Performance 20

Find this presentation and more on [Link]
[Link]

8. Advantages and Disadvantages

• Failures can be detected and • A set with too many indicators

eliminated. will tend to clutter the
• Indicators can be adapted to overview.
the circumstances. • Measuring indicators is a long-
• It is not necessary to measure term exercise. If the
every single parameter to get indicators are not defined
a “bigger picture” over the properly, and if measuring is
whole progress. not done correctly and
conscientiously, it will not
work.

Using Indicators to Measure Progress and Performance 21

Find this presentation and more on [Link]
[Link]

9. References
Business Dictionary. URL: [Link] [Accessed:
04.12.2013]

MDF (Editor) (2005): MDF Tool: Indicators. Ede: MDF Training and Consultancy. URL:
[Link] [Accessed: 04.12.2013]

MUGA, H.E.; MIHELCIC, J.R. (2008): Sustainability of Wastewater Treatment Technologies. In: Journal of Environmental
Management 88, 437 - 447. URL: [Link] [Accessed: 23.10.2013]

TAYLOR, P.; LIDEN, R.; NDIRANGU, W.; JIN, L. (2008): Integrated Water Resources Management for River Basin
Organisations – Training Manual. Pretoria: International Network for Capacity Building in Integrated Water Resources
Management (Cap-Net). URL: [Link] [Accessed:
23.10.2013]

UKAID & UNITED STATES INSTITUTE FOR PEACE (n.y.): Indicator Module. URL:
[Link] [Accessed: 10.12.2013]

WUP and WSP (2003): Better Water and Sanitation for the Urban Poor. Good Practice from Sub-Saharan Africa. Kenya:
Water Utility Partnership (WUP) and The World Bank Water and Sanitation Program (WSP). URL:
[Link]
[Link] [Accessed: 13.09.2013]

ZEWO (Editor) (n.y.): Outcome and Impact Assessment in International Development. Zewo Guidelines for Projects and
Programmes. Zurich: Schweizerische Zertifizierungsstelle fuer gemeinnuetzige, Spenden sammelnde Organisationen
(ZEWO). URL: [Link] [Accessed: 23.10.2013]

Using Indicators to Measure Progress and Performance 22

 “Linking up Sustainable Sanitation,
Water Management & Agriculture”

SSWM is an
initiative
supported by:

Created
by:

Using Indicators to Measure Progress and Performance 23