PROJECT MONITORING & EVALUATION

BY

Misgina Asmelash (Ph.D.)

Asst. Prof. of Agricultural Economics and Management at ECSU │Migration and
Development misgie2008@[Link]
Brainstorming
Brainstorm for five minutes and discuss with the person seated next to you on the
following questions.
What is project? Why projects?
What is project management?
Why PM&E?
Context
Context … Why M&E?
Context… The Power of Measuring Results
OVERVIEW OF PROJECT MANAGEMENT
>
• Simply put, a project is a series of tasks that need to be completed to reach a
specific outcome.
• A project can also be defined as a set of inputs and outputs required to achieve
a particular goal.
• Projects can range from simple to complex and can be managed by one person
or a hundred.
• Another aspect of project is the non-routine nature of activities.
• Each project is unique in the sense that the activities of a project are
unique and non routine.
• A project consumes resources.
• The resources required for completing a project are men, material,
money and time. 7

7
Comprehensive definition
Project is a temporary endeavor involving a connected sequence of
activities and a range of resources, which is designed to achieve a specific
and unique outcome, which operates within time, scope, cost and quality
constraints and which is often used to introduce change (Lake, 1997).

8
Key concepts:

• Purpose-the basic reason for the existence of a project- to solve a

problem, address a need or take the advantage of opportunity.

• Temporary: means that a project is something that has a specific start

date and a specific end date.

• Sequences of Activities: the works and the steps we perform and the
methods and knowledge we use to achieve the project objective.

9
Key concepts…
• Resources: A project utilizes a variety of resources [human, financial, material,
information, etc…] to carry out the activities or tasks.
• Unique Outcome: A project brings about a unique product or service -
something that has not existed in the organization here-to-fore.
• Scope- the extent of the problem or opportunity that the project needs to
address.
• Organization: is vital to coordinate resources to achieve the project objectives-
organizations can be public, private or NGOs.

10
Key concepts…
• Time: any project should be time bounded-it has a start and end time

• Cost: activities consume human, financial and material resources.

• Quality: the project needs to produce quality products to maximize the

satisfaction of the users.

• Introduce change: A project is often used as an instrument for change -

change for the betterment of the society.

11
Projects range in size, scope, cost and time from mega international
projects costing millions of dollars over many years to small domestic
projects with a low budget taking just a few hours to complete.

12
 Characteristics of Projects
• Objectives: A project has a set of objectives or a mission. Once the objectives are
achieved the project is treated as completed.

• Life cycle: A project has a life cycle. The life cycle consists of five stages… i.e. conception,
planning and organizing, implementation, commissioning, M&E stages.

• Uniqueness: Every project is unique and no two projects are similar..

• Team Work: Project is a team work and it normally consists of diverse areas.

• Complexity: A project is a complex set of activities relating to diverse areas.

13
Cont’d…
• Risk and uncertainty: Risk and uncertainty go hand in hand with project.

• Customer specific nature: A project is always customer specific.

• Change: Changes occur through out the life span of a project as a natural
outcome of many environmental factors.

• Optimality: A project is always aimed at optimum utilization of resources for the

overall development of the economy.

14
Examples of projects
• Build low cost/housing dev’t Projects
• Build a dam for hydroelectric power/irrigation
• Improve the standard of a road
• Set up a regional university/college
• Introduce a new (experimental) crop
• Introduce a new drug
• Create a network of village health centres
• Create a network of agric. extension services
• Create associations of municipalities
• Support the creation of tourism facilities
• Improve clean water supply
• Create wireless Internet access in universities 15
Reflection

What is the difference between a project and operational /business as usual

activities?

How plans, programs and projects are interrelated to each other?

16
Projects Vs. Operations
• Organizations perform two types of work: project work and operational
work
• Operations are ongoing and repetitive while projects are temporary and unique.

• The purpose of a project is to attain its objective and then terminate whereas the
objective of an ongoing operation is to sustain the business.

17
 Feature Projects Operations
Purpose Attain objectives and terminate Sustain the organization

Time Temporary Ongoing

Unique product, service, or result Non-unique product, service,
Outcome
Key Differences

or result
Dynamic, temporary teams formed to Functional teams generally
meet project needs aligned with organizational
People
Generally not aligned with structure
organizational structure
Varies by organizational structure Generally formal, direct line
Authority Generally minimal, if any, direct line of authority
of
Manager
authority

18
 Project Family Tree
A project normally originates from a national plan or corporate plan. In normal scheme of things, the family
tree for a project would be as given below
Plan = National/Corporate plan with target for growth.

Programme = health programme, educational programme, R&D programme.

Project = Power plant, hospital, housing project etc.

Work Package = Water supply, power supply and distribution package.

Task = Award of water supply contract, construction & foundation.

Activity = Excavation, laying of cable, preparation of drawing.

19
• Projects exist as components in broader developmental plans, their
identification, formulation and implementation should be linked to
short, medium and long term national development plans.

• Sound development plans require good projects, just as good projects

require sound planning. The two are interdependent.

20
Project Vs. Program
• A program is an assortment of related/associated projects that are managed
together to achieve a number of objectives.
• Program may also contain elements of ongoing operations. Since programs
comprise multiple projects, they are larger in scope than a single project.
• A program is a collection of projects – together all the projects form a
connected package of work.
• The different projects complement each other to assist the program in
achieving its overall objectives.

21
 Project Program
Narrow in scope Wide in scope; can comprise many
projects as components.
Differences

Specific and detail Comprehensive and general

More precise and accurate in its Broader goal related to sectoral policy
objectives and features

Possible to calculate the costs and Difficult to calculate costs and returns
returns
Similarities

• Have purpose/ objectives

• Require input (financial, manpower, material)
• Generate output (goods and/or services)
• Operate over space and time
Examples
• Vision:
Freeing the Ethiopian People from poverty and seeing economically strong and
prosperous Ethiopian people
• Goal:
Improving the living standard of the Ethiopian People within ten years
• Strategy:
Home Grown Economic Programs/………
• Plan: SDPRP, PASDEP, GTP, 10 Years Perspective Development Plan
• Program:
Resettlement /Villagisation Program…

23
Cont’d …
• Projects:
Education projects (opening elementary and secondary schools),
Road construction project,
Health projects such as establishing clinics and hospitals,
Water and sanitation projects,
Electricity, etc that are essential for a new village.

24
#Reflection
• What is project management?
• What is the difference between project Management and
general management?

25
 Defining Project Management
Project management is an organized venture for managing projects, involves:
o scientific application of modern tools and techniques in planning, financing,
implementing, monitoring, controlling and coordinating unique activities or task
produce desirable outputs in accordance with the determined objectives with in
the constraints of time and cost.
According to PMBOK,
Project management is the application of knowledge, skills, tools and techniques to
project activities in order to meet project requirements [PMI, 2008].

26
Defining Project Management…
• Project requirements- refer to the whole purpose of the project –this could be to solve
a problem, address a need, or take advantage of an opportunity.
• Knowledge- refers to as acquiring an understanding of the body of knowledge in nine
areas: integration, scope, time, cost, quality, contract, resources, communication and
risks.
• Skills – an ability or aptitude to perform something well
• Tools and techniques- tools refer to PM templates and checklists to follow, and
techniques describe how to use the tools or procedures to accomplish a specific
activity or task.

27
Defining Project Management…

• Its origins lie in World War II, when the military authorities used the techniques of
operational research to plan the optimum use of resources.

• One of these techniques was the use of networks to represent a system of related
activities

28
What are public projects?
• A public project is any project that is funded by a government, and is meant to
be owned or operated by that government.
• Most public projects relate to work a government does to fulfill a public purpose,
and commonly they include such things as road repair and construction, public
building construction, schools, and even public parks.
• Public sector projects are funded by taxpayers, and therefore are subject to more
open procedures than many other projects. For example, a public project may
need to publish requirements and request bids. Those bids must be opened at a
public place and then considered publicly.

29
Project Vs. General management
• General management is a broad subject dealing with every aspect of
managing an ongoing activity.
• Project management is a carefully planned and organized effort to
accomplish a specific objective in relatively short period of time. For
example, construct a building; construct roads, implement a new computer
system, preparing a new curriculum, etc.
• General management provides the foundation for building project
management skills and is often essential for the project manager.
• A project, which is unique and temporary, requires different management
disciplines.
• Project management is a method for getting something done, whereas
general management is a method for keeping things going.
30
 PM Life Cycle
• What is project life cycle?

• What are the stages in the project life cycle?

31
The Project Life Cycle
• PLC….is the stages through which the project passes from inception to its
completion.
• Is a continuous process made up of separate and complementary stages
(phases) each with its own characteristics and each setting a ground for the
next one.
• There are different approaches:
• European Commission /EC/
• Project Management Institute /PMI/
• WB/UNEP/NEPAD

32
The project cycle /PMI Model/

33
The Project Cycle /World Bank/
Identification

Evaluation Preparation

Proposal
development

Implementation Appraisal

Financing
decision
34
Identification Potential projects emerge from specialists, local leaders and national development
strategies. Identification of potential stakeholders, particularly primary stakeholders.
Carry out problem assessment and decide upon key objectives. Assess alternative
strategies for meeting objective.

Preparation and analysis The technical, institutional, economic, environmental, and financial issues facing
the project studied and addressed —including whether there are alternative
methods for achieving the same objectives.
Assessing feasibility as to whether and determining whether to carry out more
advanced planning. Project plan developed which can be appraised.

Appraisal Critical review or independent appraisal of project plan.

Implementation and The project plan is implemented over a specified time period.
monitoring Monitoring of project performance with a management information system to
enable correction of implementation problems as they arise.

Evaluation On-going and final assessment of the success of the project against original
objectives, to learn lessons to help improve future projects.
35
PCM refers to a process
oriented project
management system
covering the whole
project cycle from
project conception to
project completion.

Monitoring & Controlling 36

OVERVIEW OF PWOJECT MONITORING &
EVALUATION
Contents
 Overview of Project Monitoring and evaluation (PM&E)
 Monitoring
 Evaluation
 Uses of M&E
 Monitoring and Evaluation System
 Steps to Developing an M&E System
Overview of Monitoring and Evaluation
• M&E is a combination of data collection and analysis and assessing/
determining to what extent an intervention has, or has not, met its
objectives.

• M&E has two essential components of project management that help

organizations assess the progress and effectiveness of their Project/programs.
• Project M&E is a process of continual gathering of information and
assessment of it in order to determine whether progress is being made
towards pre-specified goals and objectives, and to highlight whether there
are any unintended (positive or negative) effects from a project and its
activities.
• Project M & E enables to identify and mitigate issues that may impact/affect
the project scope, quality, timeline, or budget. It also enable to take those
insights and use them to optimize processes for future project.
 Overview of Monitoring and Evaluation (M & E)

• Project M & E is an integral part of the project cycle and of good project
management practice.
• Project M&E is a continuous management function
• to assess if progress is made in achieving expected results,
• to spot/identify bottlenecks in implementation and
• to highlight whether there are any unintended effects (positive or negative)
from an investment plan, programme or project and its activities.
 Overview of Monitoring and Evaluation

• M & E focuses on the measurement of the following aspects of an intervention:

priorities of M & E
• On quantity and quality of the implemented activities (Outputs: What do we do?
How do we manage our activities?)
• On processes inherent to a project or programme (Outcomes: What were the
effects /changes that occurred as a result of your intervention?)
• On processes external to an intervention (Impact: Which broader, long-term
effects were triggered by the implemented activities in combination with other
environmental factors?)
 Why M&E

 M & E is carried out for different reasons. The major can be summarised as
follows.
• M&E results in better transparency and accountability
• M&E helps to catch problems early
• M&E helps ensure resources are used efficiently
• M&E helps organizations learn from their mistakes
• M&E improves decision-making
• M&E helps organizations stay organized
• M&E helps organizations replicate the best projects/programs
• M&E encourages innovation
• M&E encourages diversity of thought and opinions
 A Successful M &E

 A successful M&E system must allocate the following:

1. Sufficient budget (for information management, participatory monitoring activities, field
visits, surveys, etc.);( usually 5-10% of the allocated budget)
2. Sufficient time (for a start-up phase that is long enough to establish the M&E system,
conduct a baseline survey, train staff and partners, include primary stakeholders in M&E,
monitor and reflect);
3. Sufficient capacity and expertise (to support M&E development, skilled and well-
trained people required for good quality data collection and analysis) for M&E. If
appropriate, external expertise in design for a baseline study and an impact evaluation
should be engaged;
4. Sufficient flexibility in project design enabling the M&E system to influence the project
strategy during implementation.
 Key Steps in Undertaking M & E

Step 1: Agree the starting point

Step 2: Identify the approach and securing the budget,

Step 3: Implement the M&E plan

Step 4: Analyze the findings

Step 5: Communicate the learning

 Key Steps in Undertaking M & E
Step-1: Agreeing the Starting point
What is the context for developing the M&E? What issue require decision at this stage?
• Decisions about M&E would be made at the earliest stage of the project. Some of the early decisions
are:
1. Define the project: Define the scope and purpose of the M & E
2. Identify:
• The key implementers (government officials, politicians, businesses, business associations,
in-country staff, consultants: local and/or international)
• The primary beneficiaries (business owners, government officials)
• Who funds the project and whether it is a multi donor intervention
• Who provides resources for M&E
• Whether there are additional partners
• Who has the skills and is available to undertake M&E work in the team / organization
3. Who should Carry Out the M&E ? Who are the key implementers?
 Key Steps In Undertaking M & E
Step 2: Identifying the Approach and Securing a Budget
• Prior to securing a budget and putting it all together into a formal plan, there are usually six
variables to consider in M&E design.
• These factors which to be considered as sources of information are
1. Questions: Identify the key questions to be asked and answered by the M & E
2. Approach: Agree the overall M & E approach ( conventional or participatory) and methodology
3. Indicators: Choose the appropriate indicators for all aspects (from input to overall
objectives, and key criteria of evaluation) of the project
4. Data Collection: Select tools and instruments for data collection and analysis for all aspects
project(from input to overall objectives, and key criteria of evaluation) of the project
5. Timeframes: Plan clear time frames with milestones
6. Resources: Identify people and other resources
COMPONENTS OF M & E
Components of Project M & E

1. Project Monitoring

2. Project Evaluation
 Project Monitoring

 What is Project Monitoring?

• Project Monitoring is defined as: the continuous, systematic and on-going process of
collection and analysis of information (data) in relation to a program or investment
primarily for project management.
• Project Monitoring provides an indication as to the extent of progress against stated
end of a project or initiative outcomes (specific objective or purpose).
• Project Monitoring is a systematic process of verifying the effectiveness and efficiency
of output/inputs of a development intervention’s implementation in order to:
• Assess progress towards results and identify insufficiencies (or gaps); and,
• Recommend corrective measures for optimising desired results.
 Monitoring

 What is Monitoring?
• Project Monitoring is a continuous process by which stakeholders obtain
regular feedback on progress towards achieving the set milestones and results
(often focusing more on process: inputs, activities and outputs).
• Project Monitoring refers to the process of keeping track of all project-related
metrics including team performance and task duration, identifying potential
problems and taking corrective actions necessary to ensure that the project is
within scope, on budget and meets the specified deadlines.
 Monitoring

 What is Monitoring?
 Monitoring is a system for continuously gathering and processing relevant
data and information in order to:
• Document result, processes, experiences
• Identify achievement of project objectives and at target-deviations timely
recognize the needs for correction
• Make steering decisions: quickly initiate corrective measures and
adjustment.
• Make available the necessary resource for these measures
 Monitoring

 What is Project Monitoring ?

Project monitoring is carried out in
(i) measuring progress of project activities against established schedules
and indicators of success,
(ii) identifying factors affecting progress of project activities,
(iii) measuring the response of the decision taken on the project activities
and its effect on the progress of project implementation, and
(iv) minimize the risks of project failure.
 Monitoring

 Thus Monitoring
1. Means keeping a track of implementation process
2. involves watching the progress of a project against time, resources/cost and
performance schedules during the execution of the project and identifying lagging
areas requiring timely attention and action
3. is defined as a management function to guide in the intended direction and to
check performance against pre-determined plans
4. means regularly checking of progress of works against the targets laid down in
order to ensure timely completion of the project
 Purposes of Monitoring

 Importance of Project Monitoring

 Monitoring is the systematic and routine collection of information from projects
and programs for the following main purposes:
• To learn from experiences, to improve practices and activities in the future. To
promote learning: To learn from experiences on the ground.
• To develop clear corrective measures and take informed decisions.
• To have internal and external accountability of the resources used and the
results obtained;
• To take informed decisions on the future of the initiative;
• To improve process planning: helps improve project performance and the
quality of achievements
 Why is Project Monitoring Important?

 Importance of monitoring:
• To assess the project results: To know how the objectives are being met. To ensure
the achievement of anticipated results while the plan executing
• To understand stakeholder’s perspectives: Through direct participation in the
process of monitoring and evaluation, learn about the people who are involved in the
research project. Understand their values and views, as well as design methods to
resolve conflicting views and interests.
• To promote empowerment of beneficiaries of the initiative.
 Steps in Project Monitoring

1. Study of the project: its schedule, resources, costs, etc,

2. Selection of the parameters to be monitored,

3. Selection of the frequency of reporting and its format

4. Collection of data for the parameters being monitored

5. Analysis of data by using appropriate monitoring technique

6. Presentation of the analysed data and reporting it to the management

7. Review of the presented data by the management for decision making

 What to Be Monitored?
/The Focus of Monitoring/
 Project monitoring focusing on inputs, activities and outputs and their contribution to outcomes.
There are three main domains of information required in a monitoring system: Input, Activities and output.
• Inputs: Resources going into conducting and carrying out the project. These could include staff,
finance, material, and time.
• Activity/ Set of activities in which project resources are used to achieve the expected and planned
result. (e.g number of workshops or number training sessions). Activities are tasks undertaken to
transform input into output
• Outputs: Immediate results or first level result: tangible or intangible (capital goods, product or
services) obtained in short term by execution of activities ( e.g. number of commodities distributed,
number trained, number of people reached or number of people served)

Product &Service
provided

Input Activities Output Outcome Impact

Immediate
result
 Monitoring Questions and the Log-frame

Impact Measuring changes at goal-level requires a longer

time frame and is therefore deal with by evaluation
and not by monitoring
Outcome
Are output leading to achievement of the outcome?
How do beneficiaries feel about the work?
Output

Activities Are activities leading to the expected output? What is causing delays or
unexpected result?
Are tasks personnel undertaken transform input to
output Is there anything
Inputs Are activities being implemented on schedule and happening that should
with in budget?
lead management to
Are finance, Human, and Material Resources modify the operation’s
Available on time and in the right quantities and implementation plan?
quality?
 Types of Monitoring

Impact

Result Outcome
Result Monitoring
Output

Activities
Implementation Implementation Monitoring
(Means and Strategies)

Inputs
 More on Types of Monitoring
1. Process Monitoring :
• Routine data is collected and analyzed in order to establish whether the project tasks
and activities are leading towards the intended project results. It answers the
questions “what has been so far, where, when and how has it been done”
• This is often referred to as ‘activity monitoring.’
• Process monitoring is implemented during the initial stages of a project as its sole purpose
is to track the use of inputs/resources, along with examining how activities and outputs
are delivered. It is often conducted in conjunction with compliance monitoring and feeds
into the evaluation of impact.
2. Technical Monitoring: It involves assessing the strategy that is being used in project
implementation to establish whether it is achieving the required results. It involves the
technical aspects of the project such as the activities to be conducted
 Types of Monitoring
3. Assumption Monitoring: Involves measuring factors which are eternal to the project which
might affect the success or failure of the project

4. Financial Monitoring: Simply refers to monitoring project/program expenditure and

comparing them with the budget prepared at the planning stage

5. Compliance monitoring: Just as the name suggests, the purpose of compliance monitoring is to
ensure compliance with donor regulations, grant, contract requirements, local governmental
regulations and laws, ethical standards, and most importantly compliance with the expected
results of the project. The need for compliance monitoring could arise at any stage of the
project life cycle.

6. Context monitoring: Context monitoring is often called ‘situation monitoring.’ It tracks the
overall setting in which the project operates. Context monitoring helps us identify and measure
risks, assumptions, or any unexpected situations that may arise within the institutional, political,
financial, and policy context at any point during the project cycle. These assumptions and risks
are external factors and are not within the control of the project, however, context monitoring
helps us identify these on time to influence the success or failure of a project.
 Types of Monitoring
7. Beneficiary monitoring: This type of monitoring is sometimes referred to as ‘Beneficiary
Contact Monitoring (BCM)’ and the need for this may arise at any stage of the project cycle.
Its primary purpose is to track the overall perceptions of direct and indirect beneficiaries in
relation to a project. It includes beneficiary satisfaction or complaints with the project and
its components, including their participation, treatment, access to resources, whether these
are equitable, and their overall experience of change. Beneficiary monitoring also tracks
stakeholder complaints and feedback mechanism
8. Organizational monitoring: As the name suggests, organizational monitoring tracks
institutional development, communication, collaboration, sustainability and capacity
building within an organization and with its partners and stakeholders in relation to project
implementation.
9. Results monitoring: This is where monitoring entwines with evaluation. It gathers data to
demonstrate a project’s overall effects and impacts on the target population. It helps the
project team to determine if the project is on the right track towards its intended results and
whether there may be any unintended impacts.
 Monitoring Questions

 Questions addressing input, activities and output level. These are

1. Are finance, personnel and materials available on time and in the right
quantities and quality?
2. Are activities being implemented on schedule and within budget?
3. Are activities leading to expected outputs?
4. How do beneficiaries feel about the work?
5. Are outputs leading to achievement of the outcomes?
 Logic Model
• Basic Logic Model

Inputs Activities outputs outcome

Planned Activities Intended Result

• Logic Model causal pattern
Then

If
Activities Outcomes
• Participants
• Funds What • Short
• Products • Intermediate
• Staff you will Outputs • Long
• Resources do
If
If Inputs Output

Then Then
Evaluation
 What is Project Evaluation?

 Project Evaluation:

 is a periodic, systematic and objective assessment of Conception, Execution and

Result ongoing or completed project, in order to determine its relevance and level
attainment of objectives, efficiency, effectiveness, impact and sustainability.

 is the periodic systematic and objective assessment of an ongoing, or completed

project, program, or policy including its design, implementation, and results
is the process used to measure/determine/ making value judgment of / the success
and impact of project using both quantitative and qualitative data.
is the periodic review of the results of a project (typically carried out at mid-term or
at completion) towards its outcomes, development goals and impact.
 What is Project Evaluation?

The aim of project evaluation is to determine the

• relevance and level of achievement of project objectives,
• the effectiveness, efficiency, impact and sustainability.
Evaluations appraise data and information that inform strategic decisions, thus
improving the project or program in the future. information gathered in relation to
these aspects during the monitoring process provides the basis for the evaluative
analysis.
 What is Project Evaluation?
 Project Evaluation:
• May be ongoing during implementation or at the end
• Conducted at important milestones
• provide in-depth analysis
• Consider results at outcome and impact level in relation to cost
• Compares planned with actual achievements
 What is Project Evaluation?

Goal • Long-term results, widespread improvement in society

(Impacts) (reduced number of people living in poverty. Consequence of
Results

agricultural program).

Outcomes
• Mid-term result, Intermediate effects of outputs on clients
(what beneficiaries achieve due to new access to services,
etc.) – e.g. greater agricultural yields.

• Short-term results - Products and services produced (what

Outputs managers or those responsible of the project do), e.g. access
Implementation

to services, awareness campaign

Activities • Activity - Tasks personnel plan and undertake to transform

inputs to outputs, e.g. meetings, training events

Inputs
• Input (what project managers and development partners put
as - financial, human, and material resources) e.g.
agricultural inputs
 What is Project Evaluation?

 Evaluation includes:
• Looking at the aims and objectives of the project (What difference did this
project set out to make? What impact should it have had?);
• Assessing the progress made towards what we wanted to achieve at the
outset;
• Looking at the strategy chosen to implement the project (Did the strategy
work? If not, why not?); and
• Assessing whether or not funds were used efficiently
 What is Project Evaluation?

 Project Evaluation focus on three fundamental questions:

1. Descriptive Questions: to show what is happening (describes the process, prevailing
conditions, organisational relationships and points of view of various stakeholders in
the program).
2. Normative Questions: to compare what is happening with what was planned
(activities, achievements, fulfilled or non-fulfilled objectives ). Could also be relevant to
resources/inputs, activities, and outcomes/outputs.
3. Cause and Effect Question: to focus on results and to try to determine to what extent
the programme is fuelling change.
 Purpose of Evaluation
 From Accountability Perspective:
• The purpose of evaluation is to make the best possible use of resources/ by the program
managers who are accountable for the worth of their programs.
• The purpose of evaluation measuring accomplishment in order to avoid weakness and
future mistakes by
- Observing the efficiency of the techniques and skills employed
- modifying and improving the scope
- Verifying whether the benefits reached the people for whom the program was
meant
• A project is accountable for achieving outcomes and contributing to development impact
 Purpose of Evaluation
Shifting in Relative Influence and accountability

• A Project management is accountable

for facilitating this transition process
• The local partners ideally take on
increasing responsibility for converting
the project’s outputs into outcomes
and, often after the project itself has
ended, for making the outcomes
contribute to broader, long-term
impacts
 Purpose of Evaluation

 From a Knowledge Perspective: the purpose of evaluation is to

• establish new knowledge about social problems and the effectiveness of policies and
programs designed to alleviate them
• Understanding people’s participation & reasons
• Helps to make plans for the future work.
 The purpose of project evaluation is to determine the
1. Relevance
2. level of achievement of
• project objectives,
• efficiency,
• impact and
• sustainability.
 Principle of Evaluation
The following are some of the principles, which should be kept in view in evaluation.
1. Evaluation should involve minimum possible costs (inexpensive).
2. Evaluation should be done without prejudice to day to day work (minimum
hindrance to day to day work).
3. Evaluation must be done on a cooperative basis in which the entire staff and the
board members should participate (total participation).
4. As far as possible, the agency should itself evaluate its program but occasionally
outside evaluation machinery should also be made use of (external evaluation)
5. Total overall examination of the agency will reveal strength and weaknesses
(agency/program totality)
6. The result of evaluation should be shared with workers of the agency (sharing)
 What to Evaluate: The 5 Key Criteria
• Project Evaluations should help to draw conclusions about five main aspects of the intervention:
Relevance, Effectiveness, Efficiency, Impact and Sustainability.

Criterion Definition

Relevance • Appropriateness. Measure according to programme objectives which

correspond to beneficiary expectations, country needs, global priorities,
partner policies and donors.
• To what extent does the program address an identified need?
• How well does the program align with government and agency priorities?
• Does the program represent a legitimate role for government?

Effectiveness • Measure a project ‘s achieved results – or in process of being achieved, bearing

in mind their relative importance.
• To what extent is the project/program achieving the intended outcomes, in the
short, medium and long term?
• To what extent is the program producing worthwhile results (outputs, outcomes)
and/or meeting each of its objectives?
 What to Evaluate: The 5 Key Criteria
Criterion Definition

Efficiency • Measure which of the project resources have been transformed into outputs
at better cost. Sometimes requires an economic analysis of different
alternatives.
• To what extent is the relationship between inputs and outputs timely, cost-
effective and to expected standards?
Impact • Assessment of long term effects, positive and negative, primary and secondary,
resulting from a programme, directly or otherwise, intentionally or otherwise.

Sustainability • Assessment of the sustainability of benefits resulting from a development

intervention after a programme’s completion. Probability of gaining long-term
benefits.
Outcome
Evaluation Questions and the logframe
 Step in Project Evaluation
 Step 1: Set Evaluation Questions
• The types of questions can be organized into several general categories

i) Effectiveness: To what extent has implementation of project activities led to the

achievement of desired project outcomes?
ii) Efficiency: To what extent were project inputs managed in a clear, responsible and
timely fashion? Were project activities carried out as planned?
iii) Relevavcy: Were the implemented activities the most appropriate to meet the intended
project outcomes? Could better outcomes have been achieved through other means?

iv) Equity: Was participation in the project equally accessible to all potential
beneficiaries? Did project benefits reach most of the intended target group or only a
small segment of the target population?
 Step in Project Evaluation
 Step 2: Identify Who Needs to Know What, and Why
• Questions should relate directly to the information or decision-making needs of project
stakeholders
• Stakeholders could include: Target group/clients/participants, Funders/investors, Board
members, Management, Staff, and Community
 Step 3: Create an evaluation plan and evaluation indicators and develop Questions
 Step 4: Briefing the concerned people about the evaluation plan and indicators

 Step 5.: Revising and elaborating the evaluation plan

 Step 6 : Initiating Evaluation
 Step 7: Utilizing / Sharing the information
 The Process of Evaluation
• There are three points in a project where evaluation is most needed Evaluation can and should be done:
before, during, and after implementation
A) Before project Implementation: evaluation is needed in order to:
• Assess the possible consequences of the planned project(s) to the people in the community over
a period of time;
• Making a final decision on what project alternative should be implemented; and
• Assist in making decisions on how the project will be implemented

B) During Project Implementation: Evaluation should be a continuous process and should take place in
all project implementation activities. This enables the project planners and implementers to
progressively review the project strategies according to the changing activity and project objectives

C) After Project Implementation: This is to retrace the project planning and implementation process, and
result after project implementation. This further helps in
• Identifying constraints or bottlenecks inherent in the implementation phase;
• Assessing the actual benefits and the number of people who benefited;
• Providing ideas on the strength of the project for replication; and Providing a clear picture of
the extent to which the intended objectives of the project have been realized
 Stages in Evaluation
1. Program Planning Stage:
• Pre-investment evaluation or
• Formative evaluation or
• Ex-ante evaluation or Early/ Formulation
• Pre-project evaluation or
• Exploratory evaluation or
• Need assessment

2. Program Monitoring Stage:

• monitoring evaluation or Ongoing/interim
• Concurrent evaluation

[Link] completion Stage

• Impact evaluation or
• Ex-post evaluation or Summative/Terminal/Final
• Final evaluation
 Types of Evaluation

 Evaluation can be categorized under different headings:

A) By timing ( when to evaluate)

1. Formative Evaluation
• Done during the program-development stages
• conducted at mid-term (also called periodic evaluations) or semi-annually (also called
process evaluation, ex-ante evaluation, project appraisals)
2. Summative Evaluation
• Taken up when the program achieves a stable of operation or when it is terminated
• are only conducted when the project has been completed.
• are also called terminal evaluation, Outcome impact evaluation, ex-post evaluation, etc.)
 Types of Evaluation

B) By Agency( who is evaluating?)

Internal Evaluation External Evaluation

• Usually Biased • Usually Unbiased
• Subjective/ Perceived lack of • Objective / Perceived objectivity
objectivity • Detailed
• Done by the management itself • Assessment by the outsider
• On going/ Concurrent evaluation • Outside-the-box perspective
• Cheaper • Expensive
• Doesn’t require collaboration • Requires collaboration
• Lack of expertise/detail • Conducted by experts/professionals
• Lack of “outside the box” thinking
 Internal and External Evaluation
 Internal Evaluation: ( project Self Auditing )
• Internal evaluation ( or otherwise monitoring, concurrent evaluation) is a continuous
process which is done at various points and in respect of various aspects of the working of an
agency by the agency staff itself i.e. staff board members and beneficiaries.
 External/ outside Evaluation: (This is done by outsiders/ Certified Management
Audit)
• Grant giving bodies in order to find out how the money given is utilized by the agency or how the
program is implemented sent experienced and qualified evaluators(inspectors) to assess the work.
• Some donors may send consultants in order to see how far the standards laid down are put into
practice
• Inter agency evaluation: In this type, two agencies mutually agree to evaluate their program by the
other agency
Internal/ External Evaluation
 Types of Evaluation
C) By Stages
Terminal Ex-Post
On going
• At the end of • After a time
• During the
or immediately lag from
Implementation
after the completion of
of a project a project
completion
 Methods of Evaluation: Tools/Techniques
 Methods of evaluation tool or techniques include
1. First hand information
2. Formal and informal report
3. Graphical representation
4. Standing evaluation committee
5. Project profile
1. First hand Information:
• One of the simplest and easiest methods of evaluation by getting first hand
information about the progress, performance, problem areas, etc. of a project from
a host staff, line officers, field personnel, other specialist and public who directly
associated with the project.
• Direct observation and hearing about the performance and pitfalls further facilitate
the chances of an effective evaluation
 Methods of Evaluation: Tools/Techniques
2. Formal/Informal Periodic Reports
• Evaluation is also carried out through formal and informal reports
I) Formal report : consists of
A) Project Status Report: From this one can understand the current status, performance,
schedule, cost and hold ups, deviation from the original schedule
B) Project Schedule Chart: This indicate the time schedule for implementation of the
project. From this one can understand any delay, the cost of delay and the ultimate loss
B) Project Financial Status Report: It is through financial report, one can have a look at
a glance whether the project is being implemented within the realistic gadget and time
 Methods of Evaluation: Tools/Techniques…
II) Informal Reports: Informal reports such as anonymous letters, press reports, complaints by
beneficiaries and petitions sometimes reveal the true natural of the project even though these reports
are biased and contains maligned information
A. Graphic presentations: Graphic presentations through display of charts, graphs, pictures,
illustration, etc. in the project office is yet another instrument for a close evaluation
B. Standing Evaluation Review Committees: Some of the organizations have setup standing
committee, consisting of a host of experts and specialists who meet regularly at frequent intervals to
discuss about problems and to suggest remedial measures
C. Project Profiles: Preparation of the project profiles by the investigating teams on the basis of
standardized
 Complementary Roles of Monitoring and Evaluation
MONITORING EVALUATION

• Clarifies program objectives • Analyses why intended results were or were

not achieved
• Links activities and their resources to • Assesses specific casual contributions of
objectives activities to results
• Translates objectives into performance • Examines implementation process
indicators and sets targets
• Routinely collects data on these indicators, • Explores unintended results
compare actual results with targets.
• Reports progress to managers and alerts • Provides lessons highlight significant
them to the problems accomplishment or program potential and
offer recommendations for improvement
 Comparison of Monitoring and Evaluation

Monitoring Evaluation
• Performance • value
• Internally • Externally
• Continuous • One-off
 Comparison of Monitoring and Evaluation
Monitoring Evaluation
Periodic and occasional review at significant
Continuous throughout the project
Timing point in project progress – end of project, mid point
of project, change of phase
Day to day activities, outputs, indicators Assess overall delivery of outputs and progress
Scope
of progress and change towards objectives and goal

Main External evaluators / facilitators, project users,

Project staff, project users
participants project staff, donors

Regular meetings, interviews, monthly, Extraordinary meetings, additional data collection

Process
quarterly reviews, etc. exercises etc.
Written report with recommendations for changes
Written Regular reports and updates to project
to project – presented in workshops to different
outputs users, management and donors
stakeholders
 Comparison of Monitoring and Evaluation
Monitoring Evaluation

Frequency On a regular basis (e.g. through quarterly By mid-term or on conclusion ( ex-post, i.e. at
progress reports and regular observation least two years after the project has ended)

Main Action Tracking Assessment

Basic Purpose Improving efficiency and adjusting the Improving effectiveness, impact and future
work plan, if necessary programming

Focus Inputs, Activity and outputs, Effectiveness, relevance, impact and cost-
effectiveness (Outcome and Impact)

Information Self evaluation, participatory evaluation, Same as for monitoring, plus external
Sources rapid participatory evaluation evaluation and interactive evaluation

Undertaken Project staff ( in conjunction with Same as for monitoring, plus external
by beneficiaries) evaluation ( commissioned by donor agencies)
Comparison of Monitoring and Evaluation
M & E System
 M & E System
 What is an M & E System?
• An M & E system:
• Covers all the work carried out before, during or after a project to define, select, collect,
analyse and use information.
• represents all the things that need be undertaken before, during and after program
implementation, in order to track and measure progress (and success) in achieving the
goal.
• is where everything comes together, from the initial selection of objectives and indicators
to the final evaluation of a project
• refers to all the indicators, tools and processes that used to measure if a project has been
implemented according to the plan (monitoring) and is having the desired results (
evaluation).
• refers to all the functions required to measure a project/plan progress and to assess the
achievement of its results. The system is usually composed of a set of results, measured
by indicators (together called the result framework) through monitoring tools and a
manual that describes the roles and responsibilities related to its functioning.
 M & E System

• A more formal definition of a M&E system is a ‘series of policies, practices and

processes that enable the systematic and effective collection, analysis and use of
monitoring and evaluation information’. Nigel Simister, October 2009.
• The M&E system would include things like
• who is responsible for M&E tasks in the project/organization,
• the intervals where data should be collected,
• how the data is collected,
• who collects the data,
• the type of database that is used for storing the data,
• the standard forms and data collection tools to be used,
• how the data is analyzed, the evaluation questions, the frequency with which an
evaluation takes place, the budget allocated for evaluation etc.,
 Objectives of ME-System

 Objectives of an ME System
• Measure performance

• Improve accountability and management of resources

• Efficiently and effectively use data

• Improve coordination with partners

• Collect complete and timely information on national effort

 Components of a Project M&E System
 The six main components of a project M&E system
1. Clear statements of measurable objectives for the project and its components.
2. A structured set of indicators covering: inputs, process, outputs, outcomes, impact, and
exogenous factors.
3. Data collection mechanisms capable of monitoring progress over time, including
baselines and a means to compare progress and achievements against targets.
4. Clear mechanisms for reporting and use of M&E results in decision-making.
5. Sustainable organisational arrangements for data collection, management, analysis,
and reporting.
6. Where applicable building on baselines and data collection with an evaluation
framework and methodology capable of establishing causation (ie capable of attributing
observed change to given interventions or other factors).
 key characteristics of an effective M&E system

The key characteristics of an effective M&E system are the following:

[Link] measures and reports on outputs that reflect the critical stated strategic objectives of the
organization;
[Link] provides clear indicators against which the organization is working, and being measured;
and that within the organization, information for the outputs being measured is available and
verifiable.
3.A good M&E system identifies the key issues and root of the problems that you want to
address
[Link] must be cost-effective for the operating unit
[Link] must be result oriented
[Link] M&E system must itself be monitored and updated regularly
[Link] must track and effectively support the policy reform process
 key characteristics of an effective M&E system
8. Provides a user-friendly means of understanding the current status of the relevant
policy

9. Provides a rationale for how future performance targets are set

10. It must make the decision making at management level easy and efficient

11. An effective M&E System must have ways to report the findings to those who can
take action and use the findings for positive change

12. It must identify the responsible persons and the finalize the actions agreed

13. An effective and very good M&E system will be the one which reports its findings in a
positive way and as constructive criticism.
 Fundamental Steps to Carry out M&E System

1. Construct a logic model/log-frame for a project/program

2. Develop an M&E plan
3. Select indicators
4. Collect information on the selected indicators
5. Analyze the information gathered
6. Compare results with program initial goals, objectives and LM
7. Share/Use the data
THE END