M&E Framework for ESD in TVET Philippines
M&E Framework for ESD in TVET Philippines
ABSTRACT
This paper advances a design of a monitoring and evaluation (M&E) framework to help determine the performance
and success of ESD (Education for Sustainable Development) program of TVET (Technical Vocational Education and
Training) as applied to micro, small and medium enterprises in the Philippines. The M&E design, which is premised
on results-based management principles, will particularly address the rubber, furniture, and plastic industry sectors.
The proposal emphasizes the need to monitor and evaluate the program to get the desired results. The formulated
design will show how performance and success of the program can be measured empirically through processes that
employ robust methodology and rigorous research techniques. It will be of value, in particular, to the ESD program
implementation of TVET as it will provide M&E guidance to businesses, communities, and institutions, especially
those linked with TVET institutions.
190 | Section 2 - TRACK SESSION 1 - Track D: Research, Monitoring and Evaluation for ESD
This study argues that the achievement of goal or aim measure of success based on what was learned from
should be the benchmark by which the success of an previous programs.
activity or endeavour must be measured. Otherwise,
if a goal or aim will just serve as a hollow rhetoric to The use of monitoring and evaluation (M&E) is a
signal the formulation or start of a collective human requisite for most of development interventions. An
endeavour, then it might be futile to have a goal at all. M&E framework is, ideally, a part of the design and
Yes, achievement of a goal per se may not be foolproof plan of development interventions (Kusek, J. and
realistic or totally attainable but determining if Rist R. 2001). Recently it is a main requirement
one’s endeavour is resulting toward the attainment in all policies, programs and projects of the World
of planned targets or program objectives would Bank (WB), Asian Development Bank (ADB) and
be a good measure to judge whether the effort is international donor institutions, such as JBIC, CIDA,
worthwhile in the first place. and USAID, among many others.
It appears that no public and private organizations The illustration in Figure 2 shows how results-
today, such as TESDA, exist without the vision- based M&E differs from the conventional way of
mission-goal thing. It is a global practice and one assessing programs. In the traditional practice, the
might be out of place if it has none. But vision and range of performance is measured from inputs-to
mission, just like goal or aim, should not remain a activities-to outputs. This system is limited because
rhetorical wish-list expression. It shall be transformed the main concern of management is the production
into a measurable endeavour that tells one results are of deliverables (outputs) from the resources (inputs)
being achieved and not just outputs being produced. that are acted upon by the processes (activities).
The results that the interventions make, however,
The mechanism for M&E is argued here as an integral are not typically assessed, such that the measure of
part of the implementation of a policy, program or effectiveness and performance is not complete. The
project. This is offered as a way of showing empirical practice is short in determining whether the planned
evidence that the ESD-TVET project, for instance, is objectives are met and achieved, and attribution as to
achieving its established targets and results. It shows whether the intervention (development project) itself
success more ‘visibly’ and identifies failure more really caused the change is ambiguous.
concretely. An adaptation from Osborne and Gaebler
(1992) puts it more in a succinct way; The results-based approach is concerned with
assessing performance and success of programs by
WHY SHOULD PROGRAM focusing on results. Results are analyzed, measured
PERFORMANCE BE MEASURED? and compared with the set goals. This includes looking
at the short term (effects), intermediate (outcomes),
and long term (impacts) results of the intervention in
Proposing a framework for determining
the identified stakeholders, as well as the change in
success and performance of ESD-TVET ‘environment’ where intervention is made.
measurement are made to show that increased tolerance is actually achieved. It does not just keep track of how
many lectures are delivered to students; how many teachers are trained; or how many schools are administered
the program, but the system analyzes and evaluates, as done through a rigorous research process that is made
well, the results of the program through measurable part of the M&E design.
indicators involving stakeholders and the enabling
environment brought about by it. On top of these, Using these principles as backdrop, the paper advances
the process can, by some measure, demonstrate that a design of M&E framework to complement the ESD
the attainment of goal is attributed or not to the P2P program of TVET as implemented by TESDA in the
program itself -and thereby establish causality. This is Philippines.
Figure 3. Sample results-framework of ‘People to People Program’ (P2P Program), an educational program designed to
increase intercultural harmony in Czech Republic (Adaptation from: Santos, R. [Link]. 2008)
192 | Section 2 - TRACK SESSION 1 - Track D: Research, Monitoring and Evaluation for ESD
Looking inside TVET in the Philippines: As in other government agencies, the practice in
What the present system for determining TESDA maybe that focal point personnel, who
success and performance of TVET programs are tasked to do other works as well, are manning
the monitoring and assessment of the program
reveal?
implementation. And normally, reality shows
that non-related tasks are the ones attended to
The TESDA organizational structure is shown in
with priority by these personnel, although the
Figure 3. TESDA is identified as the sole policy-making
designated job should be carrying out the research,
body of TVET in the Philippines. However, based on
implementation and maintenance of the M&E.
the published hierarchy of roles and functions, there
Apparently, a specifically designed M&E system is
seems to be a weak link between implementation
not in place to complement the implementation of
and determination of performance in the program.
the program. This makes determining performance
The existence of Research and Evaluation Division
ambiguous, at best, inconsistent.
(RED) in both the Planning Office (PO) and the
TVET System Development Office (TSDO) does
An M &E system (or as used in this paper, framework)
not guarantee a strong M&E system that is in place in
built specifically for the ESD program, or even
implementing the ESD program.
Figure 4. TESDA organizational structure. The TESDA Board is the highest TVET policy-making body in the
Philippines (Adaptation from: Sijuco, A. 2005)
TESDA Board
RATIOANALIZED
STRUCTURE TESDA Board
(as of September 2005) Secretariat
Office of the Director General TESDA needs an M&E SYSTEM in place and an M&E Office
(Cabinet Rank) mandated to determine performance and provide
feedback to the stakeholders and proponents
Internal Audit
Division (IAD)
Office of the Deputy Director General Office of the Deputy Director General
Sectoral TVET (ODDG-ST) Field Operations (ODDG-FO)
Planning Office (PO) Regional Coordination Office (RCO) Office of TESDA Technology Institute
(OTTI)
• Office of the Executive Director (OED) • Office of the Executive Director (OED) • Office of the Executive Director (OED)
• Policies and Plans Division (PPD) • Policy Program Coordination Division • TESDA Technology Institute Manpower
• Research and Evaluation Division (PPCD) Operation Division (TTIMOD)
(RED) • Field Operations Monitoring Division
(FOMD)
TESDA Women’s Center
TVET System Development Office
(TSDO) Regional Offices (ROs)
National TVET Trainer’s Academy
• Office of the Executive Director (OED)
• Programs and Systems Development
Division (PSDO) Provincial Offices (POs)
• Research and Evaluation Division Specialized Centers (SCs)
(RED)
•
Qualifications and Standards Office Technology Institutions (TIs)
(QSO)
RESULTS: EFFECTS /
INPUTS ACTIVITIES OUTPUTS RESULTS: IMPACTS
OUTCOMES
- Lecturers - Lectures to - Trainees trained -Enhanced access -Increased national and
trainees - Trainees to and utilization of regional capacity to
-Trainers certified TVET programs with provide young people
-Training of - Well-equipped courses grounded access to TVET and
-Course trainees & capacitated on environmentally expanded employment
Materials techno-voc responsive practices opportunities in the
-TVET providers rubber, furniture and
- Increased quality of
capacity - Well-equipped plastic industry sectors.
-Budget TVET programs, with
building & capacitated
improved capacity
initiatives national-regional -Highly improved quality
of graduates to
TVET agencies of life of TVET recipients
address environmental
- Formulate and partners in the 3 sectors, with
sustainability issues,
enabling - Roadmap for increased evidence of en-
through enhanced
SAMPLE policies & devt -plans & vironmentally- responsive
standards, testing &
strategies strategies practice of each trade.
accreditation and
evaluation procedures.
194 | Section 2 - TRACK SESSION 1 - Track D: Research, Monitoring and Evaluation for ESD
• Provide feedback on performance and success of to achieve -placed in strategic position relative to
the program to the stakeholders, based on a set of other components of the intervention, such as inputs,
criteria identified by ESD-TVET, among others. activities, and outputs.
These functions are hinged on the goals or impacts The Design Matrix in the succeeding tables (Tables 3,
that ESD-TVET has targeted to achieve, such as; 4, 5 and Table 6 in the Addendum) illustrate a more
• Improved Access and Equity in TVET detailed process of crafting the evaluation questions
• Improved Assessment and Certification that will guide the whole M&E system. The sample
is of a quasi-experimental design approach. Only
• Decent and Productive Employment
samples are shown here and refinements can be done
• Enhanced Employability of TVET Graduates in the event of actual design formulation -which
• Improved capacity of the people to address shall be carried out as a team effort with TESDA
environment and development issues proponents.
• From these expressions of the ESD-TVET
aspirations, the Theory of Change model or Descriptive questions are used to elucidate on the
Logic Model is derived and shown in the matrix extent and scope of intervention, and the change it
of Table 2. brings about (Table 6, Addendum). The question “To
whom was the program provided?” for instance, can
give information on how extensively the intervention
The Theory of Change model is a logical representation
was delivered to the recipients. Sub-questions, such
of the aspired menu of changes or the ‘envisioned
as, among others, “How many trainees accredited?”
future’ that the ESD-TVET program intervention
show the extent by which the program facilitated the
is designed to accomplish. Changes include the
accreditation of recipients. This manner of deriving
effects in people or the community, and the ‘enabling
information, as part of the M&E system, ensures
environment’ where change can take place. The
that empirical evidence of performance are obtained
model emphasizes the results that the program wishes
rigorously.
Table 3. Design Matrix for TVET in the Philippines: ESD in the 3 sectors
Measures &
QUESTION SUB-QUESTION TYPE Target Baseline
Indicators
2. Is the ESD- Does the program conform Norm Checklist of Yes N/A
TVET program to DepED/ TESDA conformity with
being delivered standards? standard
to national /
international TVET international
standards? standards?
Design: This can be time-series comparison. Program was designed to meet standards at inception, however the
DepEd/TESDA standards may change over time.
3. Is the ESD-TVET Is delivery of the program by Norm Checklist of Yes N/A
program being TTIs & others, conforming to conformity with
delivered with organisation standards? internal standard
good practice? documents,
Normative questions (Tables 3) are made to show On the other hand, cause and effect questions (Tables
if the intervention program is carried out according 4 & 5) are crafted to seek evidence that the observed
to standards and the acknowledged criteria of changes are attributed to the program. This is called
performance. This part of the M&E system gives attribution or proof of causality. This provides
information to the proponents on how the program evidence that the program is the one that caused the
is performing and achieving its targets. Questions, change and not other factors outside the intervention.
such as “Is the ESD-TVET program being delivered
to national / international standards?” can determine
how the intervention is administered relative to
accepted norms and benchmark standards.
Table 5. Design Matrix for TVET in the Philippines: ESD in the 3 sectors
The M&E Matrix designed hypothetically in Table 7 & 8 (Addendum) presents a framework for continuous
Measures &
QUESTION SUB-QUESTION TYPE Target Baseline
Indicators
What is the number of cases Census of all
of environmentally-responsive TVET -ESD techno-
practice among grauates, voc providers;
before and after the program; C&E and a sample Yes Yes
and in contrast to the of comparison
comparison group, before and providers
5. Is there evidence after?
of environmentally-
After program completion, Survey: yearly
responsive
do techno-voc providers’
behavioural change
administrators believe that the
and practice as C&E Yes Yes
program has made a difference
a result of the
in trainees’ behaviour &
program?
practice?
Compared to before the Focus group –
programs, what is the randomly selected
evidence of increased activities C&E trainees/ techno-voc Yes Yes
& practices identified as providers
environmentally-responsive
SAMPLE 4/4
196 | Section 2 - TRACK SESSION 1 - Track D: Research, Monitoring and Evaluation for ESD
monitoring of the program. It serves as a detailed guide management applications make possible detailed
for M&E personnel to keep track of the performance documentation, easier data organization and retrieval,
of the ESD-TVET implementation. Acting as and provide a secure platform of communication and
performance monitoring framework (or plan), it is a information exchanges among team members.
matrix that combines in one structure the essence of
the (a) Logic Model, which is a framework that shows Figure 5. ICT has a high potential in establishing a
the results in relation to other components of the credible M&E system for ESD-TVET
intervention, (b) the Design Matrix, which presents
the (research) evaluation questions, and (c) other
M&E elements, such as the performance indicators
and its types; baseline data and targets; data sources;
data collection frequency and methods; person/
office in charge of collection; and reporting strategy,
among others. The matrix, which is an improved
variance of the Logframe, is a flexible framework that
can be expanded to include some other particulars of
M&E, such as units of measure, data disaggregation,
or estimated cost of collection, among others.
(Adaptation from: Lozari, E. 2007)
The above presentation on the basic building blocks of
an M&E framework for ESD-TVET implementation Program Monitoring and Evaluation: A well-
is just an initial step towards formulating a credible designed database system can deliver organized data
monitoring and evaluation system. Further works for one or multiple uses, facilitating comparative
are needed to build on these basics. An M&E system measurements and monitoring over a period of time.
that can help TESDA determine the performance and ICT can provide significant assistance to TVET
success of its programs based on empirical evidence is instructors in the assessment process of a training
possible through this process. course – software may be developed to administer,
score, evaluate, and record performance results for
ICT and M&E System for ESD-TVET objective course examinations. These records help
ICT can be an indispensable tool for the M&E system instructors to monitor the students’ progress, as
designed for ESD-TVET programs. Its use can extend well as allow program developers to identify trends
from the design and planning of the training programs, in order to improve teaching. Similar tools may be
to its implementation and management, up to the used in a greater scope for physical or digital (online)
monitoring and evaluation and presentation of the data gathering (e.g. surveys, student assessment of
results of the program. Figure 4 illustrates graphically trainers), statistical measurements and monitoring,
how ICT can help ESD- TVET implementation in database management, and institutional research.
TESDA. Implementation can be harmonized and
Program Design Collaboration Tools: ICT
systematized comprehensively through the use of
collaboration tools facilitate the process of creating
organized system of data retrieval and management.
a training program or curriculum. ICT has proved
This has a high utility level especially in the M&E
useful in gathering and consolidating teams of
system. Performance data collecting can be
stakeholders in order to construct a clear strategy and
harmonized not only in national and sub-national
work process for training, as well as documenting the
(regional) levels, but across institutions, sectors and
design process. The design of ESD-TVET programs
programs (Lozari, E. 2007).
can include the design of support ICT platforms and
Project Implementation and Management: ICT can tools. Common ICT tools include computer-assisted
revolutionize the project management process of or computer-based instruction; another approach
ESD-TVET programs. Web based-based tools are is the online or distance-learning program which
especially valuable in speeding up communication, requires that the conduct of the training and the
lowering relative communication costs, and transfer of information and instructional materials be
broadening learners’ access to information. Project web-based.
The framework is proposed for implementation of Lozari, E. 2007. Result Based Management and
ESD-TVET in micro, small and medium industry SMED Plan Monitoring & Evaluation System. The
sectors, namely: rubber, furniture, and plastic industry Private Sector Promotion Program, SMEDSEP,
sectors. However, the paper pointed out that a need Philippines.
for a comprehensive M&E system in TESDA exists.
Patton, Michael Q. 2001. “Evaluation, Knowledge
The paper also highlighted the great potential Management, Best Practices, and High Quality
contribution of ICT in supporting the establishment Lessons Learned,” 22(3): 329-326,.American Journal
and implementation of a specific M&E system for of Evaluation.
ESD-TVET.
Rist, Ray and Stame, Nicoletta, ed. 2006. From
As a recommendation, an independent evaluation Studies to Streams: Managing Evaluative Systems,
of the ESD-TVET Program can be done by utilizing Comparative Policy Evaluation, vol. 8, NJ: Transaction
an external body of evaluators, aside from an Publishers.
M&E system that is integrated within the TESDA
management domain. This will enable determination Santos, R. B. 2010. “A Look Inside Development: The
of performance and providing evidence on the Monitoring and Evaluation of Foreign-funded Urban
achievement of results through independent means. Development Projects in MM, Philippines”. 15th
This process, however, was not covered in this paper. Inter-University Seminar on Asian Megacities, March
11-13, 2010, Tokyo University, Tokyo, Japan.
198 | Section 2 - TRACK SESSION 1 - Track D: Research, Monitoring and Evaluation for ESD
Santos, R. B., et. al. 2010. “Bridging up the North TVET Strategic Plan 2011-2015, Millennium
and South Divide: An application of the theory of Development Goals (MDG).
change model in evaluating the impact of the North &
South Rail Linkage Programme, Luzon, Philippines”. Technical and Vocational Education and
8th Annual Conference on Architectural Research Training for the 21st century, UNESCO and ILO
& Education (ACARE 8), September 12-13, 2008, Recommendations, 2002.
Bulacan State University.
Zorzi, Rochelle, Martha Mcguire, and Burt Perrin.
Santos, R. B. 2010. “Comparing measures of success: 2002. Evaluation Benefits, Outputs, and Knowledge
The monitoring and evaluation (M&E) of Japan’s Elements. Canadian Evaluation Society. Report
ODA-funded projects in the Philippines, Sri Lanka available at: http//[Link]/
and Indonesia”. The Sumitomo Foundation, Japan. [Link].
BRIEF BIODATA
Mr. Ian Fulgar is the President and CEO of CreatingSkies Inc., a business and technology
consultancy firm, as well as Ivertise Marketing and Web Technologies Corp, Philippines.
He serves as the Corporate Secretary of the Pilipinas Monitoring and Evaluation Society
(PMES). He is an independent thought leader and strategist on emerging technologies,
with over a decade of executive business expertise on information and communications
technologies, social media, interactive platforms, and marketing. He has been involved in
executive facilitation with commercial and nonprofit organizations in such diverse fields
as business-to-business companies, training and human resources consultancies, consumer products, and the
academe. (Email: jifulgar@[Link])