DATA COLLECTION

Almost all monitoring and evaluation work involves data collection. There are many ways of collecting
data. Data can be collected through basic tools, such as interviews, observation and surveys. This data can
be analysed in a variety of different ways. More complex approaches combine different processes of data
selection, collection and analysis within a defined tool or methodology.

Almost all monitoring and evaluation (M&E) work involves Indeed, part of the skill of an M&E practitioner (or adviser)
data collection. Data can be collected though many is to identify which tools and methodologies are
different mechanisms. Within M&E, there is little appropriate under which circumstances, and then to adapt
consistency in the terminology used to describe these them as required to fit the situation. The aim of this section
mechanisms. Common terms include tools, templates, of the M&E Universe is to provide some background and
methods, processes, approaches, techniques, frameworks information on the many different tools and methodologies
and methodologies. However, these terms are not very well available, to help M&E practitioners make these choices.
defined. Within the M&E Universe, for the sake of
simplicity, the following terminology is used.
Selecting tools and methodologies
 A tool is a simple method for collecting data. Tools
include interviews, focus group discussions, Many different factors influence decisions over which tool
observation, photography, surveys and questionnaires. or methodology to select for an M&E task. Some of these
Tools may be used in many different circumstances, and are described below (see Bakewell, et. al., 2003).
the information they generate can be analysed and
used in many ways. Tools may also be called methods or  The nature of the project or programme: Each project
processes. or programme is different, each with its own budget,
goals, objectives and ways of working. A large project
 A template is an electronic or manual form used to or programme involving thousands of beneficiaries,
record data. It usually consists of standard questions or spread across wide geographical areas, does not
headings with space to record answers or require the same tools or methodologies as a project
measurements. based in one small community.
 A methodology is a set of tools and processes that  The stage of the project or programme cycle: Some
combine to form a consistent approach to data tools or methodologies, such as Outcome Mapping, are
selection, collection, analysis and use. Examples of designed for use during design and planning stages.
methodologies used in M&E include Randomised Others, such as MSC, can only be used after enough
Control Trials (RCTs), Outcome Mapping (OM), Most time has elapsed to see change.
Significant Change (MSC) and process tracing. Most
methodologies incorporate the use of multiple tools  The key questions that need answering: Some tools
and templates. Methodologies may also be called and methodologies are designed to identify the
techniques, approaches or frameworks. changes that have happened; others are better at
explaining how and why change has occurred. For
example, an RCT may show the level of change
Using tools and methodologies in resulting from a project or programme, but
information generated through qualitative tools such
M&E as interviews or focus-group discussions (FGDs) may
When INTRAC runs an M&E training event it always asks help explain how or why that change happened.
participants to state their expectations at the start. In  The sector and type of work carried out: In some
many cases, participants hope to leave with new M&E tools sectors of work, such as health, education and
or methodologies, which they believe will immediately agriculture, it is common to use industry-standard tools
improve their M&E capacity. They often leave or methodologies. For example, industry-standard
disappointed. This is for a number of reasons. Firstly, there tools can be used to measure the results of medical
are no tools or methodologies that are appropriate in all interventions, such as eye operations or nutrition
circumstances. Secondly, most tools and methodologies programmes. Other tools and methodologies are
need to be adapted to the context, and cannot be used ‘off designed to be used when carrying out specific types of
the shelf’. Thirdly, the value of any tool or methodology is work. For example, organisational capacity assessment
almost always dependent on how well it is used and tools (OCATs) are frequently used to monitor capacity
applied. development work.

© INTRAC 2017
 Whether quantitative or qualitative analysis is appropriate, and under which circumstances. The next task
needed: Some tools and methodologies, such as is then to adapt the selected tools or methodologies to the
surveys, are more appropriate for generating statistical particular context.
data and findings. Others, such as FGDs, are better at
generating in-depth qualitative information.
 The degree of desired stakeholder participation: Tools and methodologies covered in
Different tools and methodologies may be selected the M&E Universe
depending on how important it is to engage people in The M&E Universe contains papers on many tools and
participatory M&E. Some tools and methodologies, methodologies that can be used for data collection. These
such as Participatory Learning and Action (PLA), are have been divided into two groups.
designed to be used in a participatory way with
stakeholders. Others, such as RCTs, need to be applied  The first group contains a set of basic tools. These can
according to rigidly defined standards, and provide less be used on their own or in combination. They generate
flexibility for participation. data which can be analysed in many different ways.
They can be implemented as stand-alone tools or as
 The time and resources available: Some tools, such as
part of wider methodologies. The tools include
interviews or photography, can be used very quickly
interviews, FGDs, observation, photography, video,
and with few resources. Others, such as large-scale
surveys, questionnaires, case studies and stories of
surveys, may require significant time and expense.
change.
 The available expertise: Many tools can be applied by
 The second group contains more complex
people with little or no M&E knowledge or training. On
methodologies for data collection and analysis. These
the other hand, complex methodologies such as RCTs
include RCTs, quasi-experimental approaches, QCA,
or Qualitative Comparative Analysis (QCA) may require
MSC, process tracing, contribution analysis, Outcome
significant expertise. Complex methodologies may be
Mapping, outcome harvesting, organisational
impractical for some CSOs if the skills and expertise
assessment tools, social network analysis, appreciative
required to design and implement them is not readily
inquiry and tracer studies.
available.
 A focus on pre-defined indicators: Some tools and Of course there are many more tools and methodologies
methodologies require accurate predictions of change than are covered in the M&E Universe. However, most, if
to be effective. Others are more capable of handling not all, are either adaptations of existing methodologies or
unexpected or negative changes. For example, a quasi- variants on a theme. Once an M&E practitioner has
experimental approach normally requires a baseline to understood the principles on which the tools and
be established, covering pre-defined indicators of methodologies covered in the M&E Universe are based,
change. On the other hand, the methodology known as he/she should find it much easier to adapt existing ones or
outcome harvesting can help capture change whether design new ones to meet different circumstances.
or not it was predicted.

As can be seen from the wide range of different influences “Every decade seems to
listed above, selecting appropriate M&E tools and produce some kind of tool,
methodologies for a development intervention is not
always easy. In some cases a combination of different tools approach or methodology that
or methodologies may be needed in order to meet multiple is considered ‘universally
requirements. This is why a one-size-fits-all tool or applicable’ – the logframe,
methodology for data collection and/or analysis is often not
a realistic proposition. Participatory Rural Appraisal,
Results Based Management
That said, many CSOs throughout the world never use
complex methodologies for data collection and analysis, and now Randomised Control
and rely mainly on simple tools such as interviews, FGDs Trials. We have yet to find
and photographs, perhaps supplemented by a few case such a tool or approach and
studies or stories of change. As stated earlier, the primary
task of an M&E practitioner (or adviser) is often to help we probably never will.”
decide which combination of tools / methodologies is

Further reading and resources

The M&E Universe contains two papers that cover basic tools of data collection and complex methodologies for data collection
and analysis respectively. These papers contain links to further papers covering each individual tool and methodology
separately. Four other papers in the M&E Universe deal with related topics: sampling; the use of secondary data sources; the
principles of data collection; and the use of ICT in M&E. These can be accessed by clicking on the links below.

© INTRAC 2017
 Complex collection and analysis
Basic tools for data collection
methodologies

Sampling Secondary data sources

Principles of data collection ICT in monitoring and evaluation

References
 Bakewell, O; Adams, J and Pratt, B (2003). Sharpening the Development Process; A practical guide to monitoring and
evaluation. INTRAC, UK.

Author(s): INTRAC is a not-for-profit organisation that builds the skills and knowledge of civil society
INTRAC organisations to be more effective in addressing poverty and inequality. Since 1992 INTRAC has
provided specialist support in monitoring and evaluation, working with people to develop their own
M&E approaches and tools, based on their needs. We encourage appropriate and practical M&E,
based on understanding what works in different contexts.

INTRAC Training M&E Universe

M&E Training & Consultancy M&E Universe
We support skills development and learning on a range of For more papers in
INTRAC’s team ofhigh
themes through M&E specialists
quality offer consultancy
and engaging and
face-to-face, For
themore
M&Epapers in
Universe
training in all aspects of M&E, from core
online and tailor-made training and [Link] development the series
M&E Universe
click the
through to the design of complex M&E systems. series
homeclick the
button
Email: training@[Link] Tel: +44 (0)1865 201851 home button
Email: info@[Link] Tel: +44 (0)1865 201851

© INTRAC 2017