Indicators in Quantitative Research
Indicators in Quantitative Research
3.5
11
Indicator Module
This module will help the user understand and use indicators in
project planning and proposal writing, implementation, and
evaluation.
THIS MODULE INCLUDES:
What is an Indicator?
Practice Distinguishing between
Why use Indicators?
Indicators
When Should Indicators be used?
Practice Revising Quantitative and
Types of Indicators
Qualitative Indicators
Indicators on different levels?
Practice Revising Output and Outcome
What are the features of a good
Indicators
indicator?
Practice Creating and Developing
Who should be involved in Creating
SMART Indicators
and Developing Indicators?
Indicators
Using Indicators
Additional Resources
Page 2 of 27
Pre-test- Check your knowledge!
Please answer the following questions. Read each statement below. If you agree with a
statement, write A = YES. If you disagree with a statement, write B = NO. Also, explain
why you believe that statement is wrong.
__________ 2. Indicators are only used during the proposal writing stage of a project.
__________ 3. An indicator is vital for every project cycle and useful for problem-
identification and measuring progress towards project goals and objectives.
Page 3 of 27
What is an indicator?
An indicator is used to answer the
question: ‘how do we know whether or An indicator is a factor or variable
how much we have achieved our that provides a simple and reliable
objective?” It refers to a measure used to means to reflect changes. An
demonstrate change in a situation, or the indicator enables us to perceive the
progress in, or results of, an activity, differences, improvements or
project, or program. It is a sign showing developments relating to a desired
where we are (current situation/actual change (output, objective, and goal).
results), the progress made so far (from In DM&E, the term indicator is
baseline), and the remaining distance compatible with such terms as
performance indicator, performance
towards achieving our objective (expected
measure, indicator of success, and
results). For example: indicator of change.
# of trainings organized on
collaboration between military
personnel and civilians since program start
# of civilians who attended the training and can cite 2 behaviour changes of the
military after the training
% over baseline of military who can identify 3 examples of military behaviour
that is in violation of human rights by end of project
Example
“Indicator” is a general term that
can be applied in many different
fields and contexts. To name a few, A smoke detector in a room is an
in conflict settings, in development indicator that detects the presence
settings, in economy, education, of smoke particles in the room,
health, governance, environmental alerting when it reaches dangerous
protection, etc. No matter what
levels. It is an indicator of smoke
kinds of fields they are used in, all
and danger. It alerts us to a
indicators have one thing in
common; they consist of specific potential fire.
information that signals change in
the field they measure and they can
be used by project managers and A thermometer is an indicator that
directors as a basis for relevant
measures temperature. When the
decision-making and/or policy
change.
temperature goes down, the scale
on a thermometer goes down. It is
an indicator of temperature. It alerts
Top of the Document us to the change of weather.
The difference between indicators, targets and benchmarks are:
Page 4 of 27
An indicator is a measure of change or progress, e.g.:
# of new projects in which residents and repatriated refugees collaborate since
the beginning of the project
Increase over baseline in % of military personnel who can cite 5 appropriate attitudes of
a military person towards a civilian
A target is a clear and specific statement of desired results for a specific indicator over a
specified period of time. It is usually specified in terms of quantity or quality, target
area, target group, and time, e.g.:
65% of high-level officials practice decision-making for common good in Morocco
by end of phase 1 of the program
16 skilled advocates of coaching as a tool of leadership and conflict resolution in
the region at end of the third year of the project
Reduction of severe and moderate malnutrition in Africa by 20% by 2012
Page 5 of 27
conducted towards the development and measurement of the indicators. Meaningful
indicators obtained from a system of data collection, analysis, findings and
recommendations can lead to management information and thus play a key role in
making management information systems operational.
3. Enhancing accountability: providing data that allows program staff, managers and
directors to hold accountable to themselves, funders, beneficiaries and all other
stakeholders.
Types of Indicators
Generally speaking, there are two types of indicators: quantitative indicators and
qualitative indicators
Page 6 of 27
Quantitative indicators are usually measures of outputs and always include a number (#,
%, $, etc.). Qualitative indicators can often include a target number, usually a %, to
indicate the percentage of change from baseline to program end in knowledge, attitude
or behaviour/skills. These are, in many cases, outcome indicators, though many projects
may also use quantitative measurement for DM&E at outcomes or impact level.
Wherever possible, indicators should be direct, meaning they describe the subject to be
measured specifically, e.g., number of radio shows, number or percent of people
listening, the percent of change from pre to post, etc. When this is not possible, an
indirect or proxy indicator may be used. Normally this is used to demonstrate the
change or results where direct measures are not feasible due to data collection
constraints or time constraints. For example, SFCG often uses general listenership
surveys to estimate the number of people who may hear an SFCG radio program, or, if
ethnicity figures are unavailable, you might use data on how many speak the different
languages spoken in a region.
Direct indicators can provide the best evidence of the condition or result they are
measuring. Interpretation of indirect indicators needs to be done in a cautious and
careful manner. However, the use of an indirect indicator can be more cost-effective
than the use of a direct one, and is better than having no evidence at all.
Input indicators reflect the resources (input) allocated to an activity, a project, program
or intervention. This is a measure of all resources including human, financial, material,
technological, and information resources, e.g.:
Amount of funds allotted
Number of local consultants available
Number of radios/TVs available for distribution
Number of days available
Page 7 of 27
Number of trainers eligible for the program
The two indicator levels usually applied at SFCG are output and outcome indicators. As
with the balance between qualitative and quantitative indicators, there should also be a
balance between output and outcome indicators. Output indicators tell us
implementation is on track. Outcome indicators tell us the desired change is on track.
Page 8 of 27
is to be measured, should be clear, practical, doable, and appropriate. Indicators need
to contain certain basic information and also be able to pass tests of reliability,
feasibility, and utility in decision making.
One of the most commonly used standards to assess an indicator is SMART (Specific,
Measurable, Achievable/Attainable, Result-oriented, and Time-bound). It is also the
criteria we use at SFCG when creating indicators in proposal writing, and in monitoring
and evaluation.
Example Indicator:
Increase the percentage of participants from the southern districts reporting an
improvement in their relationship with other groups to the point where they now enter each
other’s homes from 20% in 2005 to 70% by 2008.
Specific:
Participants reporting improvement in their relationship with the other(s)
Measurable:
Percentage of participants
Achievable/Attainable:
From 20% of the participants in 2005 to 70% of the participants in 2008
Result-Oriented:
Improved to the point where they enter each others’ homes
Time-bound: SMART Indicators Example #2
Between 1 January 2005 and 1 January 2008
Example Indicator:
Increase the number of men and women participating in at least two inter-community activities
from 75 men and women/year in 2005 to 450 men and women/year for all ten program
communities before the end of 2007
Specific:
The number of men and women participating in at least two intercommunity activities
Page 9 of 27
Measurable:
Number of men and women
Achievable/Attainable:
Sometimes we will see some other criteria for good indicators which are very similar to
SMART. See the SPICED and the CREAM criteria illustrated below.
Who
should be involved in Creating and Developing
Indicators?
Keep in mind that the following varies by project, context, and personnel:
Page 10 of 27
The donors: Sometimes donors will provide us with a set of indicators before a project
starts, e.g., USAID. In this case, we will need to use, add to or adapt the indicators from
the donors in project design, monitoring and evaluation. If the donors do not provide us
with indicators, it is a good idea to involve them in the process where possible.
The project manager, DME coordinator and other staff, along with relevant
stakeholders, will make decisions about the goals, objectives, and activities of the
project. Consultation of mission plans and interviews with project staff may be
necessary to select indicators. Often, this process can help reveal differences in
assumptions of varying members of staff. Everyone should be on the same page
regarding how the program will contribute to peace and development.
The proposal writer will need to clearly describe and discuss the selected indicators in
the proposal and should therefore be familiar with the underlying premises of the
project.
The DME Specialist in the DC office can be used as a resource to help the team develop
and decide on indicators.
1. Preparation
Clarify goals, objectives and intended results for your project of interest;
Articulate your theory(ies) of change as well as the types of change specified
in the objectives;
Check what indicators have already been developed, tested, and refined in
your program and other programs with which you are in contact. You can ask
other practitioners in peacebuilding for their indicators.
2. Generation
Brainstorm all related things or dimensions that can be counted, measured,
or sized and look for creative ways to combine some of those;
Consult the parties or stakeholders in the conflict what they consider to be
significant signals of change;
Break issues into smaller components. Rather than measure reconciliation,
consider its components: mercy, justice, truth, and peace. Try not to use
vague “buzz” words. Be able to describe in operational terms what,
reconciliation, mercy, etc. will look like; have a descriptive criteria. To
Page 11 of 27
measure capacity, focus on skills, technical knowledge, process, motivation,
and opportunity. This practice also goes by the name of factor analysis where
all factors that influence the change are identified and, where possible,
weighed according to the degree of influence each factor has;
Develop a list of possible indicators.
4. Testing Indicators
Test your new and newly modified indicators for their utility in decision
making as early as possible, preferably during the design phase while there is
still time to make changes. This test is different from a test of the data
collection method. The idea is to determine the utility of the indicators in the
analysis and subsequent decisions. Pick hypothetical extremes using fictitious
data and consider how the different extremes will influence decisions. If
vastly different information has no influence over the decisions, the indicator
is probably not useful and should be changed. Select the best indicators for
your use.
You should also test both the most promising indicators and those which are
also viable but not as ideal to ensure there is an alternative in the event that
the ideal indicators don’t survive the test;
You can also use the Levels of Conflict Transformation Framework. In peacebuilding and
conflict resolution settings, sometimes people follow an organizing framework proposed
by Lederach, et al. (2007) when creating and developing indicators. Lederach, et al.
comments that as social conflict can cause changes in four dimensions: the personal, the
relational, the structural and the cultural, peacebuilding and conflict resolution work
should also try to act at each of these levels and achieve results at all four levels (as
illustrated by the diagram below).
Page 12 of 27
The organizing
framework
proposed by
Lederach, et al. is
certainly a choice
we may adopt in
our work to guide
our questions and
design for
indicators. Actually
the SFCG Nepal
Office has been
using this model in
their proposal
writing and
monitoring system.
The information
below is provided by the SFCG Nepal Office explaining the use of the Lederach, et al.
framework and how their office managed to use the Lederach, et al., framework to plan
their work. To know more about the Lederach, et al. framework, the Further Research
section at the end of the module.
Page 13 of 27
What has changed?
How do we know whether we are
working with key sectors of society?
The Cultural Overcome What has changed?
Level stereotypes and How do we know whether we are
cultural constraints working to overcome stereotypes which
that led to conflict led to conflict?
What has changed?
How do we know whether we are dealing
with cultural issues in a sustainable
manner?
Below are some examples of indicators the SFCG Nepal Office have created and
developed in their work using the Lederach, et al. framework.
Personal Objective: To increase ownership and local participation in the transition to peace
and democracy
Indicators:
Increase the % of people trained who demonstrate knowledge in peacebuilding
Increase the % of people trained who incorporate our methods in their activities
The # of peacebuilding initiatives that result from our programs
Relational Indicators:
Increase the % of citizens in target VDCs who report cooperation from across
dividing lines
The # of initiatives resulting from our programs led by people from across dividing
lines
The # of projects initiated by participants which aim to break down dividing lines
Page 14 of 27
Page 15 of 27
consideration should be given as to whether and how the indicators could be
adapted to suit the objectives of the new project or population in the new context.
For each outcome indicator, a baseline is necessary. This is crucial in measuring
progress toward an intermediate result or objective. Depending on the type of
indicator being measured, the baseline data can be a point-in-time observation, a
cumulative or an average value over a period of time. For output indicators we do
not compare with a baseline, but with our benchmarks or targets.
Indicators must be chosen and used in a way that relates very specifically to the
objectives of the project or program.
When selecting indicators, be sure to select at least one or two for each objective.
Do not rely on a single indicator to measure the significant effects of a project
because, if the data for that one indicator became unavailable, it would be difficult
to document the effects. Thus it is important to try to select a diversity of indicators
in DM&E. However, you probably need no more than two or three indicators for
each outcome.
It can be cost-effective and time-effective to build on existing indicators when
developing new indicators. However, they must be adapted to fit, or supplemented
with others that are more locally relevant.
Sometimes an indicator applied to an entire population will hide the fact that there
are variances and discrepancies among different groups in that population. Thus it
may be important to collect information on an indicator separately for each group.
This is called “data disaggregation.” Typically, information is collected in ways that
reflect the components most relevant to the project such as gender, identity,
ethnicity, age, or area of origin. e.g., if we are considering youth involvement in
politics, an increase in the number of youth joining political parties may conceal the
fact that young women are not becoming more involved.
Indicators can tell us that a change we are interested in is occurring, but they
cannot explain why or how that change occurs. Determining why change happens
requires additional research and analysis. This is because the activities a project
implemented may be the sole contribution to the change happening, or to some
degree, or not at all. There could be other factors at work. Therefore, a more
credible approach is to identify and acknowledge all the major contributing factors
and illustrate how the program contributed along with the other factors. This implies
some analysis and understanding of the many forces bearing on the issues under
consideration.
Page 16 of 27
Summary
Practice Exercises
Answers are provided below, but try getting as far as you can in the
examples before looking at the answers!
Page 17 of 27
Exercise 1:
Distinguishing between Indicators
The table below lists in its left column some indicators used in SFCG proposals. Go
through these indicators. Think what types or levels of indicators they are in the four
choices given below (Quantitative Indicators, Qualitative Indicators, Output Indicators,
and Outcome Indicators). Then put a “√” in the corresponding column you choose in the
table. Note that you may need to put more than one “√”in the corresponding columns
for each indicator. When you are finished, click here or scroll down to the next page to
see the answer sheet.
Table 1
Indicators Quantitative Qualitative Output Outcome
Indicators Indicators Indicators Indicators
# of leadership coaching training sessions
provided for decision makers in Kenya in
2008
Increase over baseline in 15% of survey
respondents in targeted areas who can
name candidates from special populations
(e.g. a woman, person with disabilities, or
from other special population) within one
month of elections
Media outlets surveyed that have
increased their elections-related
programming by two or more
programs/articles each week between
beginning of project and end of election
period
# of public fora/educational events to
teach public about peace process
supported by USG assistance from June
2006 to December 2007
Reduced % of reported acts of violence
over baseline per month as reported to
community police stations
Increase in 12% of trained youth leaders
who can name three conflict resolution
strategies, basic rights according to the
UN Declaration on Human Rights, and
citizen roles in civil society by EOP over
baseline
Page 18 of 27
Practice Exercise 1 – Answer 1
Here is the answer sheet for Table 1. Compare it with your own answer sheet to see
how many correct choices you made. For the wrong choices you made, can you
understand now why the indicator(s) is/are not the type or level you chose?
Page 19 of 27
Practice Exercises
Exercise 2:
Revising Quantitative and Qualitative Indicators
Below you will see two tables (Table 2 and Table 3). Each table uses the same three
project objectives. You will see three draft quantitative indicators in Table 2 and four
qualitative indicators in Table 3 corresponding to the objectives. None of these
indicators are SMART. Considering the objectives, can you revise these indicators to
make them SMART? Write your answers in the blank column in each table below. When
you are finished, click here or scroll down to the next page to see suggested revision for
Table 2 and Table 3 by SFCG.
Table 2
Draft Your Revised
Objectives Quantitative Quantitative
Indicators Indicators
Increase in solidarity More former enemies to
between 450 former join mixed-community
enemies in five organizations
municipalities in
Chalatenango over three
years
Enhance capacity of Number of disputes
regional and local
government institutions and
communities to monitor,
report, and manage conflict
in two years in three
southern provinces
Increase the political More women to be elected
empowerment of women in parliamentarians
the province over two years
Table 3
Page 20 of 27
Draft Your Revised
Objectives Qualitative Qualitative
Indicators Indicators
Increase in solidarity More former enemies
between 450 former identifying themselves as
enemies in five members of the larger
municipalities in community rather than
Chalatenango over three belonging to one group or
years faction
Enhance capacity of Authorities to believe more
regional and local often that monitoring
government institutions and reports lead to a timely
communities to monitor, intervention and the
report, and manage conflict prevention of escalation
in two years in three
southern provinces
Increase the political More women
empowerment of women in parliamentarians believing
the province over two years that their voices are making
a difference in decision
-making
Page 21 of 27
Practice Exercises - Answer 2
Below is a suggested revision for the draft quantitative and qualitative indicators.
Compare it with your own revision and think how you may be able to write SMART
quantitative and qualitative indicators of your own next time. Note that there is not one
answer to each revision. Different people might come up with different ways for
revision. It depends on goals and objectives of the project and the context where the
project is implemented. All indicators need to be SMART and they need to suit the
objective, population, condition and context of the project they are developed for.
Page 22 of 27
Objectives Draft SMART
Qualitative Qualitative
Indicators Indicators
Increase solidarity between More former enemies % of former enemies who,
450 former enemies in five identifying themselves as at the end of year one,
municipalities in members of the larger routinely identify
Chalatenango over three community rather than themselves as members of
years belonging to one group or the larger community
faction rather than belonging to one
group or faction
Enhance capacity of Authorities to believe more # of instances at each
regional and local often that monitoring level where authorities
government institutions and reports lead to a timely believe that monitoring
communities to monitor, intervention and the reports lead to a timely
report, and manage conflict prevention of escalation intervention and the
in two years in three prevention of escalation
Southern provinces. over the course of the
project
Increase the political More women 10% increase in women
empowerment of women in parliamentarians believing parliamentarians’ belief that
the province over two years that their voices are making their voices are making a
a difference in decision difference in decision
-making -making
Practice Exercises
Page 23 of 27
Exercise 2:
Revising Output and Outcome Indicators
Below you will see two tables (Table 4 and Table 5). Each table uses the same two
project objectives. You will see two draft output indicators in Table 4 and two outcome
indicators in Table 5 corresponding to the objectives. None of these indicators are
SMART. Considering their objectives, can you revise these indicators to make them
SMART? Write your answers in the blank column in each table below. When you are
finished, click here or scroll down to the next page to see suggested revision for Table 4
and Table 5 by SFCG.
Table 4
Objectives Draft Your Revised
Output Indicators Output Indicators
Increase trust between the Training events
two communities.
Increase inter-community Public policy debates
collaboration on public or forums
policy issues that address
common interests.
Table 5
Objectives Draft Your Revised
Outcome Indicators Outcome Indicators
Increase trust between the More mobility among
two communities. residents living on both sides
within the areas
Increase inter-community More public policy debates
collaboration on public or forums among the three
policy issues that address communities
common interests.
Page 24 of 27
Below are suggested revisions for the draft output and outcome indicators. Compare it
with your own revision and think how you may be able to write SMART output and
outcome indicators of your own next time. Note that there is not one answer to each
revision. Different people might come up with different ways for revision. It depends on
goals and objectives of the project and the context where the project is implemented.
All indicators need to be SMART and they need to suit the objective, population,
condition and context of the project they are developed for.
Page 25 of 27
Top of the Document
Additional Resources
The following resources are particularly good for the development of
goals and objectives.
Anne J. Atkinson and Carolyne Ashton, Planning for Results: The Safe and Drug-Free
Schools and Community Program Planning and Evaluation Handbook (Richmond:
Page 26 of 27
Virginia Department of Education, 2002), accessed 24 September 2010,
[Link]
Cheyanne Church and Mark M. Rogers, Designing for Results: Integrating Monitoring
and Evaluation in Conflict Transformation Programs Part 1 (Washington DC:
Search for Common Ground, 2006), accessed 24 September 2010,
[Link]
Department for International Development, Log Frame Guidelines (London: DFID, 2005)
SPICED
(Roche, 2002, [Link]
47D5-8A14-1E1833901BFE_BBA5D8DC-5C40-4F9C-A6A4-0268098134D7_1.htm
[accessed 15 September 2010])
Page 27 of 27