Chapter Four: Project Evaluation

Evaluation Terms of Reference

The terms of reference (ToR) document defines all aspects of how a consultant or a team will
conduct an evaluation. It defines the objectives and the scope of the evaluation, outlines the
responsibilities of the consultant or team, and provides a clear description of the resources
available to conduct the study. Developing an accurate and wellspecified ToR is a critical step in
managing a high-quality evaluation. The evaluation ToR document serves as the basis for a
contractual arrangement with one or more evaluators and sets the parameters against which the
success of the assignment can be measured.

The specific content and format for a ToR will vary to some degree based on organizational
requirements, local practices, and the type of assignment. However, a few basic principles and
guidelines inform the development of any evaluation ToR.

―Terms of reference‖ refers to the document that details an assignment for an individual
evaluator or team of evaluators. This same phrase can also be used in the plural (ToRs) to refer
to multiple such documents.

The ToR is typically developed during the planning phase for an assignment and is often used to
attract and hire an evaluation consultant or team through a competitive process. The document
then serves as the basis for a contractual arrangement between the commissioner of an evaluation
and the external consultant(s) or in-house staff carrying out the work.

A ToR presents an overview of the requirements and expectations of the evaluation. It provides
an explicit (clear) statement of the objectives of the evaluation, roles and responsibilities of the
evaluators and the evaluation client, and resources available for the evaluation. A ToR provides
clearly detailed parameters for:—

1. Why and for whom the evaluation is being done

2. What it intends to accomplish

3. How it will be accomplished

4. Who will be in involved in the evaluation?

5. When milestones will be reached and when the evaluation will be completed

6. What resources are available to conduct the evaluation?

ToRs are needed for a broad range of evaluation-related activities, regardless of the scope of the
evaluation, budget, or timeline. A ToR can cover an entire evaluation operation or just a subset
of tasks. For example, the document might focus on the in-depth analysis prescribed to address
one evaluation question. However, a ToR should be as brief and concise as possible. A typical
length is 5–10 pages, with administrative annexes supplementing this information where
appropriate, depending on the complexity and depth of the evaluation.

When Is a Terms of Reference Needed?

ToRs can guide the work to be completed in all phases of a project or program. A ToR focused
on evaluation-related activities is typically developed for the following types of work:

• Needs assessments

• Prefeasibility studies

• Feasibility and design studies

• Comprehensive evaluations, including performance audits and impact evaluations

• Individual evaluation tasks

• Appraisal missions

• Activity or program reviews

• Independent activity completion reports

• Technical advisory groups

• Audits.

ToRs are critical for determining the working arrangement between the commissioner of a study
and an external consultant or team. They are also beneficial for in-house arrangements to clarify
scope, responsibilities, schedules, and budgets.

Content of the Terms of Reference

The text of the ToR should provide sufficient background information related to the assignment,
and then move in logical order from the evaluation objectives and intended users through to the
required qualifications of the consultant or team and the resources available. The level of detail
and ordering of specific sections will vary based on the nature and magnitude of the evaluation
task but standard topics are typically covered.

Formative and Summative Evaluations

The terms ―formative evaluation‖ and ―summative evaluation‖ were first introduced by Michael
Scriven, a prominent philosopher of science, in his book ―The Methodology of Evaluation‖
published in 1967.
Scriven used the term ―formative evaluation‖ to refer to evaluation activities that are conducted
during the development process of a program or product, with the aim of improving its quality.
The term ―summative evaluation‖ was used to refer to the evaluation of the program or product
at the end of the development process, with the aim of determining its overall effectiveness.

Since then, the terms have been widely used in various fields, including education, psychology,
public health, and business, to describe the two types of evaluation methods.

Formative evaluation is an ongoing process that involves collecting feedback and data during
the development of a program, product, or service. The purpose of formative evaluation is to
identify areas for improvement and make changes as needed to ensure that the final product or
service meets the intended goals and objectives. Formative evaluation is usually conducted
during the early stages of development and is often used to guide decision-making and improve
the quality of the final product.

Summative evaluation, on the other hand, is conducted at the end of a program, product, or
service to determine its effectiveness and success in achieving its goals and objectives.
Summative evaluation is usually conducted using predetermined criteria and involves collecting
data through surveys, tests, or other methods to measure the outcomes of the program. The
purpose of summative evaluation is to provide feedback on the overall effectiveness of the
program, product, or service and to inform decisions about its future.

Formative evaluation is typically conducted during the development or improvement of a

program or course. Summative evaluation involves making judgments about the efficacy
(effectiveness) of a program or course at its conclusion.

Importance of Formative and Summative Evaluation in project (Program) Development

Formative and summative evaluations play a crucial role in the development of effective
programs. Here are some reasons why they are important:

1. Identify areas for improvement: Formative evaluation allows program developers to

identify areas for improvement during the development process. This helps to ensure that
the final program meets the intended goals and objectives and is of high quality.

2. Provide feedback: Both formative and summative evaluations provide feedback that can
be used to improve the program. Formative evaluation provides ongoing feedback that
can be used to make adjustments during the development process, while summative
evaluation provides feedback on the overall success of the program.

3. Measure effectiveness: Summative evaluation measures the effectiveness of the program

in achieving its intended goals and objectives. This information can be used to determine
whether the program is successful and to inform decisions about its future.
 4. Ensure accountability: Evaluation helps to ensure accountability by providing evidence
of program outcomes. This information can be used to demonstrate program effectiveness
to stakeholders and funders.

5. Improve decision-making: Evaluation provides data-driven information that can be used

to inform decision-making. This helps to ensure that decisions are based on reliable and
valid information, rather than assumptions or opinions.

Formative and summative evaluations are important for ensuring the success of programs. They
provide feedback, measure effectiveness, ensure accountability, and improve decision-making.

Managing Evaluations
Managing evaluation starts firstly with gaining consensus on when, what and how to evaluate the
program, project or policy. Whose role it is to lead this process really depends on the
organizational context and structure. However, monitoring and evaluation (M&E) professionals
should be at the forefront of raising awareness about the role and value of evaluations, and
working towards creating awareness on evaluation needs in the organization.

Managing an evaluation involves agreeing on how decisions will be made for each cluster of the
evaluation (from framing an evaluation to reporting and supporting use) and ensuring they are
implemented well.

As you work through the process of planning and implementing the evaluation, you may need to
revisit and revise the choices you have made.

In this section

 Understand and engage stakeholders

Identify who has an interest in the evaluation in addition to the primary intended users, and
whose interests need to be prioritised and why. Ensure their engagement throughout the
evaluation.

 Establish decision making processes

Specify how decisions will be made about the evaluation—who will provide advice, who will
make recommendations, and who will make the actual decisions.

 Decide who will conduct the evaluation

Clarify who will actually undertake the evaluation. This might include people who are involved
in what is being evaluated (such as implementers, clients and community members), an internal
or external evaluator, or some combination of these.
  Determine and secure resources

Identify what resources (time, money, expertise, equipment, etc.) will be needed and available
for the evaluation. Consider both internal resources (e.g. staff time) and external resources (e.g.
participants' time to attend meetings to provide feedback).

 Define ethical and quality evaluation standards

Clarify what will be considered appropriate quality and ethical standards for the evaluation and
what will need to be done to ensure these standards are achieved.

 Document management processes and agreements

Develop any formal documents needed, including a brief and a Terms of Reference.

 Develop planning documents for the evaluation or M&E system

Develop a formal plan which sets out how an individual evaluation or a range of M&E activities
will be undertaken.

 Review evaluation (do meta-evaluation)

Decide processes to review the evaluation process, findings, and conclusions drawn.

 Strengthen evaluation capacity

Choose ways of building on and developing capacity for managing, undertaking and/or using
evaluation, which might involve human capital (knowledge and skills), organizational capital
(such as technical and administrative infrastructure) and social capital (supportive networks).

Writing an effective Monitoring and Evaluation report

Make sure your report is concise and the layout clean and consistent. Focus on results and
accomplishments and link the use of resources allocated to their delivery and use. Be sure to
include a section describing the data sources and data collection methods used so that your
findings are objectively verifiable.

Guidelines for writing credible and constructive monitoring and evaluation (M&E)
reports.

The following guidelines are the most commonly used monitoring and evaluation report
writings:-

1. To help ensure efficiency, the purpose of reporting should be clearly defined. Be sure
to include a section in the introduction describing the need to produce this report and its
anticipated use.
 2. Make sure the information you are providing is accurate, complete, reliable, timely,
relevant and easy to understand.

3. Be clear who your audience is and ensure that the information is meaningful and useful
to them. If needed, tailor the content, format and timing of the report to suit the
audiences’ needs. Information is of little value if it is too late or infrequent for its
intended purpose.

4. Consistency is key. Reporting should adopt units and formats that allow comparison
over time, enabling progress to be tracked against indicators, targets and other agreed-
upon milestones.

5. Make sure your report is concise and the layout clean and consistent.

6. Focus on results and accomplishments and link the use of resources allocated to their
delivery and use.

7. Be sure to include a section describing the data sources and data collection methods
used so that your findings are objectively verifiable.

8. Write in plain language that can be understood by the target audience. Avoid complex
jargons and details if possible and be consistent in your use of terminology, definitions
and descriptions of partners, activities and places. Be sure to define any technical terms
or acronyms in the annex section.

9. Make use of graphs and charts to communicate your findings. Present complex data
with the help of figures, summary tables, maps, photographs, and graphs that are easier to
understand.

10. Include references for sources and authorities.

11. Include a table of contents for reports over 5 pages in length.

12. Make sure your reporting system is cost effective. Avoid excessive, unnecessary
reporting. Information overload is costly and can burden information flow and the
potential of using other more relevant information.

13. Be open to feedback. Make sure to include an email address, a physical address or a
telephone number for the recipients to send their feedback on the report.

As we can see, there are many benefits of good and timely reporting and it should be a part of
every development project and its monitoring and evaluation system. However, many
organisations continue to report on their progress only as a part of the donor requirement,
without giving much heed to the huge prospect of collaboration, learning, adaptation and
improvement. Therefore, a good reporting system should have a balance of all these elements,
plus quality project management and good coordination and communication flow within the
team.

Reporting Project Progress and Findings

In short, a project progress report details the work your team has already completed toward your
project’s objectives and deliverables. This includes updates on your project’s timeline, tasks
you’ve completed, budget status, risks, and roadblocks your team has encountered.

Your audience for this type of report often varies from internal stakeholders to the client
themselves.

Who’s in charge of creating progress a report?

Project managers are usually the ones who collect, analyze, and organize relevant data into
readable reports. Presenting and sharing this information helps ensure the project is advancing on
schedule and meeting the budget.

Having a project manager periodically create and send progress reports is vital to keep all of your
stakeholders on the same page throughout the duration of the project. These high-level overviews
offer valuable insight into the status, allowing those involved to know whether the project is on
track or at risk.

What are the components of a project progress report?

Throughout the report, you’ll want to make sure that you’re providing valuable and easily
digestible information that your audience cares about.

Before creating a report, place yourself in the shoes of your reader and ask yourself these
questions:

 Is your audience anticipating learning certain information?

 Are they expecting a specific level of detail and depth?

 What is most important for them to understand?

As mentioned earlier, what’s included in each of your status reports will depend on the type of
project and what information is most useful. In general, though, the following components are
included.

Key Performance Indicators (KPIs)

Key Performance Indicators — or KPIs — track performance throughout the project life cycle.
They provide an at-a-glance look into the project's progress and help evaluate its success.
Common project management KPIs include time, budget, scope, customer satisfaction, and
more. KPIs could also be specific to your industry or project.

You don’t want to overwhelm your audience by filling your report with unnecessary information,
so try to include only the most relevant data. If needed, provide a way for stakeholders to review
additional KPI metrics outside of the progress report.

Milestones

Milestones mark crucial points in the timeline and status of a project. They typically mark events
like the start and end of a project phase or the completion of critical tasks.

Progress reporting tools make it easy to identify milestones that have been completed and those
which remain uncompleted. Teams use this information to determine where delays may have
occurred and adjust plans accordingly.

If certain milestones were supposed to be completed by a particular date, this report would show
just how far behind you may be.

Current project status

Above all, your progress report should provide a current and up-to-date view of the project
status. This helps your audience know what is happening with the project, making it easier to
identify potential issues in the progression of tasks and take appropriate action when and if
needed.

General project status can be a simple color indicator. The RAG (red, amber, green) color model
is often used to quickly describe the project status.

 Red: The project is stalled and needs attention

 Amber: The project is proceeding but perhaps not on budget or on schedule

 Green: The project is on track as planned

Task lists

Including a task list in your progress report can help your audience better understand which tasks
are left to complete as well as which are already checked off.

Ideally, the task list would include the team member responsible for completing each task, the
expected due date, and any other relevant information.

Project costs
Project costs can quickly add up, especially if teams do not have proper cost controls in place.
Any major discrepancies between planned and actual expenses should absolutely be noted in
your project report. In this way, the project scope or budget can be altered if necessary.

Not only will this but closely leave you with valuable budget insights you can use when planning
your next project.

Project risks

Effectively managing projects and staying on track requires thorough identification of project
risks. This should be done early on during the planning phase of your project, although
oftentimes risks pop up after the project is already underway.

A risk to a project is anything with the potential to threaten your project’s success, such as your
access to a particular resource or the availability of a certain team member.

In the risk section of your project reports, you’ll want to include the likelihood of the risk
happening, its implications, and a contingency plan in case it does happen.

Project timeline

Keeping an eye on a project’s timeline is a key part of project success. A timeline provides a
high-level overview of progress and whether the project can be completed before the deadline.

If things aren’t going according to schedule, your report should include a comparison of the
actual and planned project timeline, as well as provide an explanation of the reasons behind the
delay to transparently show stakeholders where you’re at.

Follow this 8 step format for progress report writing to ensure you include all the
important details:

 Place identifying details at the top. ...

 Project details. ...

 Summary of the report. ...

 Core activities. ...

 Current quantifiable results. ...

 Challenges encountered. ...

 Recommendations and suggestions. ...

 Concluding paragraph and signatures.