ABET Accreditation Assessment at Qassim
ABET Accreditation Assessment at Qassim
net/publication/269270027
CITATIONS READS
3 281
2 authors:
Some of the authors of this publication are also working on these related projects:
Porosity measurement of Iron Oxide using Computer Vision System View project
All content following this page was uploaded by Al-Badrawy A. Abo El-Nasr on 28 December 2015.
Abstract—In this paper, a continuous improvement Engineering at Qassim University initiated external
process was developed and implemented in our evaluations conducted by the ABET. This process was
engineering programs at College of Engineering, Qassim planned initially to make sure that what our programs
University, Saudi Arabia. The continuous improvement offer, is equivalent to their peer programs worldwide,
plan was based on an integrated set of strategies aimed
at: establishing and implementing a structured process
in addition, to satisfy the graduates and society needs.
that translates educational objectives into measurable Accreditation, as it is known, is a voluntary certified
outcomes, and specifies feedback tracks for corrective mechanism for engineering programs, administered by
actions. As a result of setting of direct and indirect recognized professional bodies [1]. It is also a positive
measures, the programs can assess and evaluate the level move towards securing professional legitimacy,
of achievement of the program outcomes during the granting accredited engineering programs their
course of the educational process. These relate to the deserving honor and esteem through continued
skills, knowledge, and behaviors that students acquire in professional development. Internationally, ABET has
their matriculation through the program. The been considered as one of the highly respected bodies
documentation of the continuous improvement process,
prepared by our engineering programs, describes two
that lead this process.
intersecting assessment loops; slow and fast. The slow In a major shift influenced by pressure from
loop refers to the process by which the program industry and global competition, ABET has
examines its program outcomes. The fast loop refers to
introduced a new Engineering Criteria 2000 (EC
the process by which we assess the degree to which our
students and graduates achieve the educational program
2000) [2], which address the effectiveness of
objectives developed in the slow loop. engineering education programs by focusing on an
assessment and evaluation process that assures the
The stages of developing and implementing the achievement of a set of educational objectives and
objectives, outcomes and the assessment plan for the outcomes. An important element of these criteria is
engineering programs are presented. Preliminary results
from pilot implementations are also presented.
the establishment of a continuous improvement
Improvements are already evident in the areas of process thorough outcomes assessment. Since then,
teaching effectiveness, assessment of student learning, the old system of counting course credits was largely
and involvement of all the constituencies. Based on these abandoned and replaced by an outcomes-based
procedures, the Accreditation Board for Engineering process. Outcomes based assessment process focuses
and Technology (ABET) visit to our engineering on what the students learned or what they actually can
programs had achieved great success and all of the do at the time of graduation. In its accreditation
programs have fully accredited on August 2010 and the criteria, ABET reaffirmed a set of "hard" engineering
next visit to our engineering programs will be on 2016. skills while introducing a second, equally important,
Keywords-Accreditation; Self Assessment; PEOs; POs; set of six "professional or soft" skills. These latter
Continuous Improvement; Soft skills skills include communication, teamwork, and
understanding ethics and professionalism and
I. INTRODUCTION engineering within a global and societal context,
Engineering education is considered to be pivotal lifelong learning, and knowledge of contemporary
in the development of modern societies. One way of issues. Engineering schools must present evidence that
ensuring and demonstrating the high quality of their graduates achieve reasonable levels of these
engineering education is through evaluation carried skills, in addition to technical discipline-specific
out by accreditation bodies like ABET. In an effort to knowledge content [3]. Measuring student outcomes
improve and maintain the quality of engineering and understanding the learning experience is critical
education offered by its programs, the College of for continuous improvement and satisfying
accreditation agencies. Furthermore, the design and this basis, the College of Engineering decided to
implementation of the educational programs should be modify its academic programs in order to be in
harmonized to achieve the departments' mission and alignment with the new system and job market
the programs educational objectives and outcomes. On requirements, taking into consideration, the
the other side, the educational program promotes recommendations raised by the engineering
continuous improvement to: enhance its ability to educational conference held by College of
satisfy the program's constituencies by meeting their Engineering, Qassim University in 2006. In 2006, the
needs and enhance the students' achievement of the Faculty of the ME department had revised the PEOs
program objectives and outcomes [4]. of the undergraduate program and finalized them to
take a final form that published in different ways such
In fulfilling ABET criteria; many hurdles have
as the student handbook [6] and appears in the college
been faced in our engineering program: the most
website [7] and in the student displaying boards in the
difficult ones were the establishment of Program
department building. This process occurred through a
Educational Objectives (PEOs) and Program
series of faculty meetings along with the available
Outcomes (POs). This has gone through number of
program constituencies including: expected employers
trials and modifications made by the Programs
of our graduates, Faculty of the ME department,
Committees (PCs) that were formed to prepare for
Professional Advisory Boards, and University
accreditation. Though, the ABET definitions [2] of
Council, and work of several committees and sub-
objectives as "broad statements that describe the
committees on program development. In addition, the
career and professional accomplishment ---", and
pioneer engineers in Saudi Arabia were asked for their
outcomes as "narrower statements that describe what
inputs on the suggested objectives. The PEOs are
students are expected to know and be able to do ---"
developed to be consistent with the mission of the
seem to be simple and straightforward, nevertheless,
College of Engineering, Qassim University, that stated
the PCs along with the main program constituencies
as: "The College of Engineering seeks to meet the
and in consultation with the Professional Advisory
needs of the Saudi society and the region with
Boards (PABs) kept revising and rephrasing the
outstanding engineering programs in education,
objectives and outcomes each time it discovered that
research, and community service" [5,7] and in
there is a need for more elaboration and/or confusion
accordance with the ABET accreditation criteria.
of the two criteria [5]. Professional expertise from
industry and the program alumni were the main tools The ME Program Committee has finally
for the assessment of PEOs and POs, while faculty formulated the following objectives for the ME
and senior exit students assessed only the program as follow. The ME graduates should:
POs. Assessment of PEOs and POs show normally the
1. Possess the skills and knowledge necessary for
strengths and weaknesses of the whole educational
work environment including design, installation,
process. Subsequently, continuous improvement
operation, maintenance and inspection of
strategies were set to elaborate on the strengths and to
mechanical and energy systems.
overcome the weaknesses. The improvement process
2. Utilize information technology; data analysis; and
should describe also the available documentation
mathematical, computational, and modern-
processes commonly used in making decisions
experimental techniques in solving practical
regarding program improvement. The implementation
problems.
of these strategies was made by revising the
3. Fit in various working environments through
curriculum, teaching methods, lab facilities … etc [5].
effective communication skills and the ability to
In this paper, the development process of the work in multidisciplinary teams.
PEOs, POs and the continuous improvement process 4. Engage in life-long learning and gain skills to
are presented. The difficulties encountered during this promote professional development, and career
process, in addition to the result of continuous planning based on social and professional ethics.
improvement conducted for mechanical engineering
It was planned to periodically collect and
(ME) program, are discussed. In the following, we
document the needs and feedbacks of the program's
will focus only on the ME program as an example of
various constituencies regarding the program
the undergraduate engineering programs of our
objectives as will be outlined in section 5. Based on
college.
these needs and feedbacks, the PEOs are evaluated to
II. ME PROGRAM EDUCATIONAL OBJECTIVES check that the established objectives significantly
(PEOS) reflect the constituencies' needs.
In 2005, Qassim University has adopted a new III. ME PROGRAM OUTCOMES (POS)
system called the Preparatory Year Program (PYP) in
The ME program outcomes describe knowledge,
which the new university students enrolled for one
skills and ability of students by the time of their
complete academic year studying math, physics,
graduation from ME department. These relate to the
English language and other general courses before
skills, knowledge, and behaviors that students acquire
they are eligible to admit to the scientific colleges. On
in their matriculation through the program. The POs
of the ME program support the PEOs and the College order to quantify this relationship, three correlation
mission. And as mentioned in the preceding section, in levels are used. A solid square indicates the strongest
2006 the College of Engineering took the opportunity correlation between a specific outcome and a given
and adapted its academic programs in order to be educational objective. The next level of correlation,
aligned with the new system that implemented in the marked by a half-filled square, represents lower level
university. At that time, the educational outcomes for correlation. An open square indicates no correlation.
the ME program were established by the Faculty of As shown in the table, every PEO is correlated with at
ME department in consultation with the available least one program outcome with strong degree of
constituencies. Then, the ME department council correlation.
formulated the POs to be closely aligned to the ABET
B. Relationship of Courses in the Curriculum to the
Criterion 3 (a) through (k) outcomes [2]. In addition,
Program Outcomes
one extra technical outcome was added to the eleven
ABET outcomes. The POs were presented to various A summary of the relationship between the
constituent groups to get their comments and review. curriculum courses and the established outcomes and
The outcomes of ME program are published on the the ME program is shown in Tables A.2 in appendix
college website [7] and listed in the student handbook A. The data in this tables indicate the level of
[6]. The outcomes of the ME Program are as below: contribution of courses in the ME curriculum to the
outcomes (a) – (k) and specific ME outcome (l). The
At the time of graduation, our students will have: course syllabi in the self study reports [5,7] show
a. An ability to apply knowledge of mathematics, clearly that each course outcome fulfills part of the
science, and engineering; requirements of the program outcomes.
b. An ability to design and conduct experiments, as IV. THE CONTINUOUS IMPROVEMENT PROCESS
well as to analyze and interpret data;
(CIP)
c. An ability to design a system, component, or
process to meet desired needs within realistic A. Development and Implementation
constraints; The outcomes assessment process means here
d. An ability to function on teams; that: "The systematic collection, review, and use of
e. An ability to identify, formulate, and solve information about educational programs undertaken
engineering problems; for the purpose of improving student learning and
f. An understanding of professional and ethical development" [4]. In our programs, the continuous
responsibility; improvement process was basically based on an
g. An ability to communicate effectively; integrated set of strategies aimed at: establishing and
h. The broad education necessary to understand the implementing a structured process that translates
impact of engineering solutions in a global,- educational objectives into measurable outcomes,
economic, environmental, and societal context; performance criteria and then specifies feedback
i. A recognition of the need for, and an ability to tracks for corrective actions, Figure 1 [8,9]. On the
engage in, life-long learning; basis of this understanding, the ME department
j. A knowledge of contemporary issues; provides the necessary assessment training on the
k. An ability to use the techniques, skills, and plan, creating an assessment tools, and identifying and
modern engineering tools necessary for reviewing key institutional practices to ensure that
engineering practice; and they are aligned with the assessment process. The
l. The ability to work professionally in both conceptual model of the assessment plan used in the
thermal and mechanical systems areas including department is shown in Figure 2. The process is
the design and realization of such systems. basically an application of the methodology known as
The program outcomes from (a) to (k) are in full "Plan-Do-Check-Act" (PDCA). This plan describes
consistence with the stated ABET outcomes Criterion two interacting assessment loops. These are often
3 [2], while program outcome (l) adopts the ME referred to as the "slow loop" and the "fast loop" [10].
specific criteria. It is planned to periodically evaluate The slow loop refers to the process by which the
the POs and revise them if needed in order to department examines its program outcomes. During
continuously improve their compatibility with the the first few years of the assessment process (2006-
program objectives and their achievability. The 2007), the ME department reviewed its program
process described in section 4 shows the methodology outcomes and course outcomes each year. After the
of periodically evaluating, revising and improving the first few years, the program and course outcomes
program outcomes. became an accurate reflection of the current program
A. Relationship of Program Outcomes to Program objectives. The program outcomes and course
Educational Objectives outcomes were then only reviewed and updated every
five years to reflect changes in the curriculum. The
A summary of the relationship between the
PEOs and POs were reviewed by ME Program
established outcomes and the objectives of the ME Committee in spring 2006.
program is shown in Table A.1 in Appendix A. In
Performance Criteria
Learning Outcome
Research and gather
Program Educational Objective information
The ME graduates should fit in various An ability to Fulfill duties of team
working environments through effective function on roles
communication skills and the ability to
team Shares in work of team
work in multidisciplinary teams.
Listens to other
teammates
Figure 1 Transferring the educational objectives to outcomes and then to performance criteria.
Mission
Feedback for
Educational Program Evaluation:
Continuous
Objectives Outcomes Interpretation
Improvement
of Evidence
Assess and
Measurable Performance Assessment:
Evaluate
Criteria Data Collection
and analysis
Program
Constituencies Educational
Practices/Strategies
Students
Graduates
TABLE 1 THE HIGHLY CORRELATED COURSES* CORRESPONDING f ME 352, ME 371, ME352, ME371,
TO EACH PROGRAM OUTCOME ME 381, ME 380 ME382, ME383
Program Highly correlated Specified courses for g ME 383, ME 384, ME241, ME352,
Outcomes courses with each the recent graduate ME 352, ME 363 ME464, ME362
POs batch h ME 372, ME 471, ME251, ME361,
GE 104, GE 202, ME251, ME361, ME 330, ME 360 ME471, ME331
a ME 371, ME 382, ME371, ME382,
ME 241, ME 251, ME372, ME341 i
ME 463, GE 105 ME 473, ME 351 ME498, PHYS104
ME 383, ME 384, ME352, ME361, j GE 104, GE401, ME ME371, ME382,
b 380 ME498, GE401
ME 352, ME 363 ME464, ME383
ME 341, ME 441, ME341, ME361, k ME 461, ME 464, ME241, ME361,
c GE 105, ME 330, ME461, ME464
ME 472, ME 473, ME441, ME461
ME 330, ME 372 ME 380
ME 384, ME 464, ME341, ME352, l ME 461, ME 464, ME241, ME341,
d ME 471,ME 473, ME ME361, ME372
ME 352, ME 363 ME361, ME382
ME 362, ME 373, ME251, ME341, 481
e * All the details regarding ME courses codes and title are
ME 481, ME 372 ME361, ME371
found in Table A.2
3) Results of the Assessment Process
To measure the achievement of the program
outcomes, the previous assessment criteria were
applied on the collected data. It is found that
significant achievement of the program outcomes
were obtained as in the following results [5]:
a) As shown in Table A.3, the percentage of
student passing the courses are more than 60%
and average final grades are more than 60%
for all the courses except for the courses ME
251, ME 362, and ME 472. These results
satisfy the first assessment criterion.
b) As shown in Figure 3. Mean rating of the
responses from the senior exit surveys
regarding each outcome are at least 3.9 out of Figure 4 The average rate of the rubrics analysis
5 for the courses specified in Table 1. applied to the outcomes achievement exam.
c) No negative comments were reported
regarding the program outcomes from the
course instructor and graduation project exam
committees.
d) No negative comments were reported
regarding the program outcomes from the
Students Advisory Board Report and
Professional Advisory Board report. On the
other hand the External Visitor had some
comments regarding the written reporting
projects.
e) The rubrics analysis applied to the outcomes
achievement exam regarding the program
outcomes resulted in an average of 2.9 out of 4,
shown in Figure 4.
f) The feedback from rubrics analysis of Figure 5 Feedback from rubrics analysis of
graduation project and summer training for the graduation project and summer training for the
graduated students regarding the program recent batch of graduated students.
outcomes resulted in an average of 2.93 out of
4, as demonstrated in Figure 5.
g) For the recent batch of graduates, the average
rate calculated using rubric analysis of the
specified courses in Table 1 is 2.83 out of 4, as
demonstrated in Figure 6.
5
TABLE A.1 MAPPING BETWEEN THE PEOS AND POS FOR THE ME PROGRAM
TABLE A.2 MAPPING OF MECHANICAL ENGINEERING COURSES TO THE PROGRAM OUTCOMES: THE TABLE INCLUDES THE REQUIRED
AND ELECTIVE COURSES.
ME Program Outcomes
Course Codes and Names
a b c d e f g h i j k l
GE 104 Basics of Engineering Drawing
Basics of Engineering
GE 105
Technology
GE 201 Statics
GE 202 Dynamics
ME 241 Mechanical Drawing
ME 251 Materials Engineering
ME 330 Manufacturing Processes
ME 340 Mechanical Design-1
Required courses in Major
Measurements and
ME 343
Instrumentation
ME 351 Mechanics of Materials
ME 352 Mechanics of Materials Lab
ME 360 Mechanics of Machinery
ME 363 Mechanics of Machinery Lab
ME 371 Thermodynamics -1
ME 372 Thermodynamics – 2
ME 374 Heat and Mass Transfer
ME 380 Fluid Mechanics
ME 383 Thermo-fluid Laboratory -1
ME 384 Thermo-fluid Laboratory -2
ME 400 Senior Design Project
System Dynamics and
ME 467
Automatic Control
System Dynamics and
ME 468
Automatic Control Lab
ME 423 Renewable Energy
ME 425 Solar Energy
Elective courses in Major
TABLE A.3 THE AVERAGE FINAL GRADES, PERCENTAGE OF STUDENTS PASSING THE COURSE AND MEAN RATING FROM STUDENT
SURVEYS OF THE SELECTED COURSES.