Chapter Five: Evaluation of Projects and Determinants of Project

Success/Failure

5.1 Indicators of project success: Time, Cost, and Quality

5.2 Criteria for project evaluation
5.2.1 Relevance
5.2.2 Efficiency
5.2.3 Effectiveness
5.2.4 Sustainability
5.2.5 Impact
5.3 Agents of project success/failure
5.4 Factors for project success/failure
Introduction
Project evaluation
• Project evaluation is a systematic and objective assessment of an ongoing or
completed project.
• The aim is to determine the relevance and level of achievement of project
objectives, development effectiveness, efficiency, impact and sustainability.

• Project evaluation and appraisal are often referred to together as project

assessment. Though appraisal and evaluation deal with similar issues, they have
a different purpose and are performed in completely different phases of a
project.

• Project appraisal is concerned with assessing, in advance, whether a project is

worthwhile and therefore if it should be proceeded with. Appraisal is done
before, and as such is a predictive/prospective action.
• It is related to defining objectives, ways to reach them, risks involved and
estimation of costs/benefits of each available option. Project evaluation is
done after, and involves analysis of the past and looking at what was done and
means to improve it.

• The process of project evaluation is concerned with assessing, in a

retrospective sense, the performance of a project after it has been
implemented and completed.

• Evaluation thus helps bring out elements of strength and weakness, success or
failure. The results are valuable in planning future projects and in attempts
to avoid repeating or committing ‘mistakes’.

• The primary purpose of evaluation in project management is to assess

performance, reveal areas where the project deviates from goals, and uncover
extant or potential problems so they can be corrected.
Common rationales for conducting project evaluation are:
• To inform decisions on operations, policy, or strategy related to ongoing or future program
interventions;
• To demonstrate accountability to decision-makers
• To enable learning and contribute to the body of knowledge on what works and what does not
work and why;
• To verify/improve program quality and management;
• To identify successful strategies for extension/expansion/ replication;
• To modify unsuccessful strategies;
• To measure effects/benefits of program and project interventions;
• To give stakeholders the opportunity to have a say in program output and quality;
• To justify/validate programs to donors, partners and other constituencies
• Recognition of actual changes and progress made;
• Two kinds of evaluation occur in projects.
1) Formative evaluation
2) Summative evaluation

1) Formative evaluation:
Happens throughout the project life cycle and
provides information to guide corrective action.

Formative evaluation is designed to pilot the

project as it progresses.

It asks the questions “What is happening?” and

“How is the project proceeding?”
2) Summative evaluation:
Occurs after the project is completed and
focuses on the end product or result.
It is designed to appraise the project after
completion.
It addresses the questions “What happened?”
and “What were the results?”
GUIDELINES FOR EVALUATION (FIVE PHASES)

A: Planning the Evaluation

B: Selecting Appropriate Evaluation Methods

C: Collecting and Analyzing Information

D: Reporting Findings

E: Implementing Evaluation recommendations

PHASE A: PLANNING THE EVALUATION
 Determine the purpose of the evaluation.
 Decide on type of evaluation.
 Decide on who conducts evaluation (evaluation team)
 Review existing information in program documents
including monitoring information.
 List the relevant information sources
Phase B: Selecting Appropriate Evaluation Methods
 Identify evaluation goals and objectives.
 Formulate evaluation questions and sub-questions
 Decide on the appropriate evaluation design
 Identify measurement standards
 Identify measurement indicators
 Develop an evaluation schedule
 Develop a budget for the evaluation.
PHASE C: COLLECTING AND ANALYSING
INFORMATION
 Develop data collection instruments
 Pre-test data collection instruments
 Undertake data collection activities
 Analyze data
 Interpret the data
PHASE D: REPORTING FINDINGS
 Write the evaluation report.
 Decide on the method of sharing the evaluation
results and on communication strategies.
 Share the draft report with stakeholders and revise as
needed to be followed by follow up.
 Disseminate evaluation report.
PHASE E: IMPLEMENTING EVALUATION RECOMMENDATIONS

 Develop a new/revised implementation plan in

partnership with stakeholders.
 Monitor the implementation of evaluation
recommendations and report regularly on the
implementation progress.
 Plan the next evaluation
 5.1. Indicators of project success: Time, Cost, and Quality

• This refers to ways to Measure the Success of Any Project.

• Of course there are many indicators of project success, but what
do you need to be measuring while the project is in motion?
• At various points during the project you want to evaluate
four points: schedule, quality, cost ,stakeholder satisfaction and
scope.
• You should be doing this informally anyway. A formal project
evaluation is of use during the end of a phase or stage as it can
give you a clear indication of how the project is performing
against the original estimates.
• This information can then be used to grant (or withhold) approval
from moving on with the next chunk of work.
How to define success criteria?
• “Project success criteria are the standards by which the project will be judged
at the end to decide whether or not it has been successful in the eyes of the
stakeholders.”

Why are project success criteria important?

• Projects ‘fail’ in the eyes of the media and stakeholders because for people are left
to guess what success looks like. Is it delivery on time? Is it delivery on budget?
Perhaps those two things really don’t matter much to the stakeholders concerned if
they get a great quality result and happy customers.
• Successful organizations take the guesswork out of this process: they define what
success looks like, so they know when they have achieved it.
1. Schedule

• Project management success is often determined by whether or not you

kept to the original timeline.

• Experienced project managers know how hard that is, but it’s a little bit
easier if you continually evaluate your progress as you go.

• Is there a hard deadline, or does the schedule relate to something else

(budget, product launch date, etc.)? In the end, did you complete the project
by the time it was due? Sometimes clients come with a hard deadline, other
times they’re simply looking for the final product. Either way, my team
always has a schedule we need to meet
2. Quality

• The end of a project phase is a good time for a quality review.

• You can check both the quality of your project management practices
– are you following the change management process every time and
so on – and also the deliverables.
• A quality review can evaluate whether what you are doing meets the
standards set out in your quality plans.
• Best find out now before the project goes too far, as it might be too
late to do anything about it then.
3. Cost

• Many executives would rate cost management as one of their highest

priorities on a project, so evaluating how you the project is performing
financially is crucial.
• Compare your current actual spend to what you had budgeted at this
point.

• If there are variances, look to explain them. You can use a project
control panel to check your actual spend in real time.
4. Stakeholder Satisfaction

• Your wider team – your stakeholders – are essential in getting much of

the work done, so it’s worth checking in with them.
• Find out how they are feeling about the project right now and what
you could be doing differently.
• This is a difficult measure to document statistically, although there’s
nothing to stop you asking them for a rating out of 10. Even if you are
evaluating their satisfaction subjectively, it is still a useful exercise. If
you notice that stakeholders are not fully supportive, you can put plans
in place to engage them thoroughly to try to influence their behavior.
5. Scope What do you need to get done within the timeframe? Tony
refers to scope as the “stars that align to bring the client, the team, and
you together.” It may be a list of features or just an idea, but the scope
should essentially be the driving force of the project.
 5.2. Criteria for Project Evaluation
5.2.1. Relevance: The appropriateness of the project objectives to
the problems it addresses and to the physical and policy
environment.
5.2.2. Efficiency: Results achieved at reasonable cost i.e how well
inputs/means have been converted into results in terms of quality,
quantity and time?
5.2.3. Effectiveness: An assessment of the contribution made by the
results to achievement of the project purpose and how assumptions
have affected project achievements.
5.2.4. Impact: The effect of the project on its wider environment.
Its contribution to the objectives for the sector (overall objectives).
5.2.5. Sustainability: Likelihood of the benefits produced by the
project to continue to flow after end of project with particular
reference to ownership, environment, policy support, institutional
capacity and financial support.
5.3 Agents of project success/failure

The following are the most Responsible for Project Failure?

1) Executive Management
2) Project Managers
3) Team Members
5.4 Factors for project success/failure
5.4 Factors for project success/failure
• For projects, there are multiple challenges to ensuring a successful
engagement. It is important to identify these challenges and
adequately plan to avoid common causes of project failure.

• Typical causes of project failure occur when the following criteria for
success are not met:
1. on time delivery,
2. on or under budget,
3. acceptance by client based on stated scope of work.
• Only a few projects achieve all three criteria. Many more are delivered
which fail on one or more of these criteria, and a substantial number
fail badly enough that they are cancelled.
• You can take certain actions which will ensure your contracts do not
fail.
22
Common Causes of Project Failure

Projects often fail because of one or more

of the following five reasons:
1. Poor planning,
2. Lack of leadership,
3. Inadequate knowledge,
4. People problems,
5. Lifecycle problems.

23
 Reason 1: Poor Planning

Poor planning can include:

− Lack of communication.
− Not breaking down development into phases or steps.
− Not prioritizing operational activities, objectives.
− Not obtaining stakeholder approval.
− No business plan or inadequate business plan.
− Unrealistic expectations set, e.g., financial investment, time
required, set-up costs.
− Inadequate funding/capital or poor use of funds/capital.
− Lack of time commitment.
− Unrealistic scheduling.
24
 Reason 2: Lack of
Leadership
Lack of leadership can include:
− Not defining ownership or the leadership structure or not
identifying decision makers.
− Not making decisions timely or decisively.
− Lacking relevant business and management expertise in areas
such as finance, purchasing, selling, production, and hiring and
managing employees.
− Neglecting your leadership role.
− Not having a strategic vision.
− Holding unrealistic expectations of others.

25
 Reason 3: Inadequate
Knowledge

Inadequate knowledge can include:

− Lacking skills and a proven approach to project
management.
− Failing to price your product or service correctly.
− Not addressing potential risks due to inexperience.
− Not estimating, monitoring, or controlling expenditures.
− Not putting a process in place for measuring and
tracking results.
− Having an incomplete or vague project work plan.
− Using inadequate control systems.
26
 Reason 4: People Problems

People problems can include:

− Lacking contact with senior management.
− Lacking leadership.
− Lacking effective project team integration between
clients, the supplier team, and the supply chain.
− Being unable to resolve conflicts.
− Not having adequate resources due to under/over
estimation of work.

27
 Reason 5: Lifecycle
Problems
Lifecycle problems can include:
− Failing to clearly and completely define the requirements,
resulting in building the wrong features or gaps in the features
needed.
− Using new or state of the art technology that cause unanticipated
problems.
− Using a poor technical design that does not allow for
modification or is not scalable.
− Changing requirements late in the project and continuing change
requests which cause the project to drift.
− Using technology components that do not fit together as
designed.
− Using poor initial testing techniques that cause repeated errors. 28
Figure 5.1. Factors affecting project Success
 How to Prevent Project Failure
(
• Require weekly status reports that include:
• Project start and completion dates.
• Which milestones you’ve passed.
• Percentage of the project that is complete.
• Any accomplishments worth mentioning.
• Important meetings attended.
• Any threats or potential risks to the projected timeline.
• Description of any problems you’ve encountered and
resolved.
• Personnel or equipment limitations.
• Budget status.

30
How to Prevent Project Failure……….
• Build an effective team by considering:
• Employee skill, experience, participation ability, the projects they
are already working on, and morale.
• Pair newer resources with mentors.
• Set a realistic schedule and stick with it.
• Establish concrete, clear goal planning in project
management.
 How to Prevent Project
Failure….
• Ensure senior management ownership and leadership from the
beginning.
• Require effective engagement with project stakeholders.
• Ensure adequate skills and proven approach to project management
and risk management.
• Pay attention to breaking developments and implementation into
manageable steps.
• Evaluate proposals based on long-term value rather than price to
secure delivery of business benefits.
• Maintain connectivity with the industry at senior levels.
• Ensure effective project team integration between clients and the
supply/resource chain.
32
 Key Takeaways from This
chapter
• For projects, there are multiple challenges to ensuring a successful
engagement.
• It is important to identify these challenges and adequately plan to avoid
common causes of project failure.
• Project failure can be avoided by:
• Planning properly.
• Hiring the right team.
• Putting the right metrics in place.
• Creating clear links between the project and the organization's key strategic
priorities.

END OF THE CHAPTER

33