Creative Education
2013. Vol.4, No.1, 71-81
Published Online January 2013 in SciRes (http://www.scirp.org/journal/ce) http://dx.doi.org/10.4236/ce.2013.41010
Copyright © 2013 SciRes. 71
Academic Program Assessment: A Case Study of a Pragmatic
Approach
Riyadh A. K. Mehdi, Mahmoud S. AbouNaaj
Ajman University of Science & Technology, Ajman, UAE
Email: ajac.mehdi@ajman.ac.ae, abounaaj@ajman.ac.ae
Received August 30th, 2012; revised September 28th, 2012; accepted October 12th, 2012
Accreditation criteria of Information Technology programs require effective learning outcomes assess-
ment and evaluation with rigorous processes, well documented results, broad faculty participation, and
complete coverage of the assessment and evaluation cycle. This paper describes a model that the College
of Information Technology at Ajman University of Science and Technology uses to implement a com-
plete outcome-based assessment and evaluation plan of its programs. The plan contains detailed accounts
of procedures and tools used to measure the achievements of program learning outcomes. Information
which is gathered from exam results, faculty, students, alumni, internship, and employers are used to
measure the level of achievement of each learning outcome from a different perspective. A final decision
is made with respect to each learning outcome. This decision is based on combining the results of the
various relevant measurement tools for that outcome. The assessment model described in this paper was
used for the successful accreditation of all programs offered by the college of Information Technology
and adopted by other colleges at the University.
Keywords: Program Learning Outcomes; Program Assessment; Course Learning Outcomes; Course
Assessment
Introduction
Academic programs assessment and evaluation is becoming
an important process in providing improved education to stu-
dents through modified curriculum and instruction. The Com-
mission for Academic Accreditation (CAA) of the Ministry of
Higher Education and Scientific Research (MOHESR) accredits
each of the programs offered by the College of Information
Technology (CIT) at Ajman University of Science & Technol-
ogy (AUST). The CAA implements standards and procedures
that require an academic program to provide an evaluation of
program quality and effectiveness as part of a self-study report.
Assessment has also become a tool of accountability in educa-
tion by providing evidence on how effective the teaching is
(ABET, 2010). An assessment plan will determine how well
students are benefiting from a learning experience offered by a
program of study.
Assessment activity at AUST started in 2001 with two online
forms filled by students. These are The Student Course Evalua-
tion Form and The Academic Advisory Evaluation Form. The
first form collects students’ feedback with regard to each course
taken during the semester. The first set of questions evaluates
the course, textbooks, and laboratory work. The second set of
questions evaluates the performance of the instructor from a
student’s point of view. The third and the fourth group of ques-
tions relates to examinations and information resources respec-
tively. The second form contains a set of questions that evaluate
the performance of the academic advisor again from a student’s
point of view. These evaluations which are useful in identifying
persistent problems in some courses or with instructors aretaken
into account in the annual evaluation of faculty members.
The next major advancement in assessment at AUST oc-
curred in the second semester of 2004/2005 when the Depart-
ment of Computer Science established procedures, and tools for
assessing and evaluating the learning outcomes of the Com-
puter Science program as part of a pilot study conducted by
AUST.
This paper describes the assessment model as it is applied to
the Computer Science Program. The model adopted the defini-
tions of program educational objectives and program learning
outcomes provided by ABET (ABET, 2010). The model has
been applied by the College of Information Technology for the
accreditation of its current programs. A similar approach was
used by other colleges at AUST and will be used for the up-
coming re-accreditation of the IT programs. It is based on a
strong course committee structure, and a detailed mapping of
program learning outcomes to course learning outcomes. This
model allows assessment to be done effectively at the courses
level so that each course can use the most appropriate assess-
ment tools. The approach is vigorous in completing the entire
assessment cycle, and it enhances faculty participation. Hereaf-
ter, the term “program” in this study refers to the Computer
Science program unless stated otherwise. The Computer Sci-
ence, Computer Engineering, Information Systems, and Multi-
media programs of the College of Information Technology have
long been accredited by the CAA. A new Information Tech-
nology Program was also accredited recently by CAA based on
the assessment model described in this work.
Literature Review
Assessment of students’ learning outcomes plays an impor-
tant role in educational effectiveness, improvement, and sus-
tainability that is increasingly being recognized and required by
R. A. K. MEHDI, M. S. ABOUNAAJ
accrediting bodies (Buzzetto-More & Alade, 2006). Assessment
is an integral part of assuring that an educational institution
meets necessary standards, as well as crucial means of provid-
ing evidence necessary for seeking and maintaining accredita-
tion (Love & Cooper, 2004). Kellough identified seven pur-
poses of assessment (Kellough, R. D. & Kellough, N. G., 1999):
1) Improve student learning.
2) Identify students’ strengths and weaknesses.
3) Review, assess, and improve the effectiveness of different
content delivery strategies.
4) Review, assess, and improve the effectiveness of program
curriculum.
5) Improve teaching effectiveness.
6) Provide useful administrative data that will expedite deci-
sion making.
7) Communicate with stakeholder.
Thomas, T. A. (1995) stated that “Assessment is an on-going
process aimed at understanding and improving student learn-
ing. It involves making our expectations explicit and public;
setting appropriate criteria and high standards for learning
quality; systematically gathering, analysing, and interpreting
evidence to determine how well performance matches those
expectations and standards; and using the resulting information
to document, explain, and improve performance. When it is
embedded effectively within larger institutional systems, as-
sessment can help us focus our collective attention, examine
our assumptions, and create a shared academic culture dedi-
cated to assuring and improving the quality of higher educa-
tion”.
Although there are differences in detail between different ac-
creditation agencies, most accreditation criteria are structured in
a similar manner. The criteria typically require programs or
institutions to:
Specify in clear terms the skills, including cognitive skills
(i.e. knowledge) that they expect students to achieve by the
time they graduate (sometimes referred to as “program
learning outcomes”).
Set up an assessment process to determine the extent to
which the program or institution is successful in enabling
students to achieve these learning outcomes.
Establish system, which implements program improve-
ments, by using data collected through the assessment
process.
Most of the other criteria are driven by the explicitly formu-
lated program learning outcomes. Thus, a typical curriculum
criterion will state that the curriculum is designed in such a way
that successful completion allows students to achieve the speci-
fied skills by the time of graduation. Atypical faculty criterion
will state that the faculty must be qualified to deliver the cur-
riculum and to revise the program in light of the data collected
in the assessment process. A typical facilities criterion will state
that the physical and library facilities are adequate to allow
students to achieve the specified skills etc. (Gowan, MacDonald,
& Reichgelt, 2006).
In the last few years, learning outcomes have achieved a
widespread importance in conferences and the literature as a
model of assessing the knowledge and skills obtained from a
learning experience. Learning outcomes have applications at
three distinct levels:
1) the local level of the individual higher educational institu-
tion for course units/modules, programs of study and qualifica-
tions;
2) the national level for qualifications frameworks and qual-
ity assurance systems; and,
3) internationally for wider recognition and transparency
purposes (Ashiem, Gowan, & Reichgelt, 2007; Ashiem et al.,
2007).
Learning outcomes focus on measurable cognitive, behav-
ioural, and attitudinal development of students as they interact
with a learning activity. They are what students are expected to
demonstrate in terms of knowledge, skills, and attitudes upon
completion of a learning experience (Adam, 2004; Ashiem,
Gowan, & Reichgelt, 2007).
Learning outcomes and outcome-based approaches have im-
plications for curriculum design, teaching, learning and as-
sessment, as well as quality assurance. They are likely to form
an important part of the twenty-first century approaches to
higher education and reconsideration of such vital questions as
to what, who, how, where and when we teach and assess
(Ashiem, Gowan, & Reichgelt, 2007). In terms of curriculum
design and development, learning outcomes are at the forefront
of educational change. They represent a change in emphasis
from teaching to learning that characterize what is known as
the adoption of a student-centred approach in contrast to tradi-
tional teacher-centred viewpoint. Student-centred learning pro-
duces a focus on the teaching-learning-assessment relationship
and the fundamental links between the design, delivery and
measurement of learning (Adam, 2004).
To implement a learning outcomes approach, program must
first formulate the program educational objectives (broad goals)
that address the institutional and the program’s mission state-
ments. The program’s mission is responsive to the expressed
interests of various program stakeholders. Then, the program
must formulate a set of program learning outcomes (knowledge,
skills, and attitudes) the programs’ graduates should have. Pro-
gram learning outcomes must directly address the educational
objectives andmay encompass certain specified learning out-
comes specified by appropriate bodies as in the case of ABET
for engineering and information technology programs. These
program educational objectives and learning outcomes must be
specified in a self-study report. The next step is to formulate a
set of measurable learning outcomes for each course in the
curriculum. Based on these courses’ learning outcomes, a map-
ping is constructed between the program learning outcomes and
courses’ learning outcomes. This mapping will be used as part
of a system to provide a quantitative measurement of the level
of attainment of each program learning outcome. This system is
based on the degree to which the learning outcomes of the cor-
responding mapped courses have been achieved.
Program learning outcomes are also assessed by using other
complementary assessment tools. Sanders and McCartney
(2003) reported a survey on twelve assessment tools used in
computer science accreditation. These tools include, among
others, senior exit surveys, alumni survey, written and oral exit
examinations, portfolio, and external advisory panel. Each tool,
by itself, has its own set of limitations and none of them is
course-based. Blanford and Hwang (2003) suggested five as-
sessment methods including class assessment, faculty interview,
and a student focus group. Class assessment is a course-based
assessment tool in which an instructor writes an assessment of
course being taught. Course assessment processes, among other
things, enablea program to demonstrate precisely how specific
program learning outcomes are addressed in the curriculum. If
course learning outcomes are then assessed continuously and
Copyright © 2013 SciRes.
72
R. A. K. MEHDI, M. S. ABOUNAAJ
the results are used to improve instruction that address them,
the degree to which the program meets its self-selected goals
must inevitably improve. In this paper, program educational
objectives are measured indirectly by mapping them to program
learning outcomes.
Course assessment can be time consuming; consequently,
minimizing faculty time is of key importance. For closing the
loop, Crouch and Schwartzman (2003) recommended forming a
departmental steering committee of senior faculty members to
consolidate all course learning outcomes into a final set of
learning outcomes. As a result, not all faculty members will
then need to participate in this step. Blanford and Hwang (2003)
suggested an assessment day as an effective way for faculty to
meet, evaluate assessment results, and provide improvement
recommendations.
Program Assessment Methodology
The Department of Computer Science hasimplemented the
following sequence of steps that are used to conduct assessment
and evaluation of program learning outcomes.
1) Establish program educational objectives that are consis-
tent with the program mission.
2) Develop measurable program learning outcomes.
3) Map program learning outcomes to program educational
objectives.
4) Design the curriculum to reflect program educational ob-
jectives and program learning outcomes and adhere time to
international standards of professional bodies at the same time.
5) Develop student learning outcomes for each of the courses
in the curriculum.
6) Create a mapping matrix between courses and program
learning outcomes, indicating which courses contribute to
which program learning outcomes.
7) Determine a reasonable number of measurable perform-
ance criteria for each program learning outcome.
8) Identify measurement (assessment) tools used to assess
the extent to which a student achieves the learning outcomes for
each course.
9) Choose program learning outcome assessment and evalua-
tion tools.
10) Develop a plan for data collection. For each program
learning outcome, identify what data is to be collected, when to
be collected, and who is responsible for collecting it.
11) Measure program learning outcomes.
12) Measure program educational objectives using the map-
ping developed in step 3 above.
13) Determine how and when the assessment information
will be reviewed and evaluated to make program improvements
(Closing the loop).
The College Assessment Committee (CAC) conducts the as-
sessment on regular and timely bases. The CAC consists of the
dean and heads of departments or their representatives. The
committee meets between semesters or upon request from the
dean. The committee has the following functions.
1) Analysis of data gathered from individual course assess-
ment tools.
2) Development and modification of assessment tools.
3) Report changes according to ACM/IEEE Curriculum
4) Report to the college council the suggested changes in the
program or course learning outcomes.
5) Review and provide recommendations to the college
council for mapping course to program learning outcomes
6) Report to college council any shortcomings of meeting
program learning outcomes.
The overall assessment plan for the Computer Science pro-
gram is depicted in Figure 1. Assessments of program learning
outcomes are detailed in the following subsections.
Program Educational Objectives an d Le arni ng
Outcomes
The Curriculum Development Committee in the Department
of Computer Science has formulated both educational objec-
tives and program learning outcomes consistent with the de-
partment mission. The curriculum was designed specifically to
allow students to achieve the intended program learning out-
comes.
Program Educational Objectives
Program educational objectives are broad statements that de-
scribe the career and professional accomplishments that the
program is preparing graduates to achieve (ABET, 2010). For
the Computer Science program offered at AUST, they can be
stated as:
1) Provide students with current core knowledge of computer
science that allows them to investigate and provide solutions to
computer science related problems.
2) Prepare students for a professional career in computer
science and related areas.
3) Offer broad and in-depth curriculum that prepare students
to pursue graduate studies or engage in life-long learning in
computer science and related disciplines.
Program Learning Outcomes
The curriculum design of the Computer Science Program at
AUST is influenced by three considerations. First, the curricu-
lum structure and areas of knowledge are based on the recom-
mendations of the Association for Computing Machinery ACM
and the IEEE Computer Society standard in order for the cur-
riculum to be of acceptable international standards (Association
of Computing Machinery & IEEE Computer Society, 2008).
The second consideration is the guidelines of the MOHESR
which stipulate that students must attain knowledge and com-
petency equivalent to completing one or more university-level
courses in each of the following area: 1) English, Arabic or
other languages; 2) the humanities or arts; 3) the natural sci-
Figure 1.
Assessment and evaluation information flow plan.
Copyright © 2013 SciRes. 73
R. A. K. MEHDI, M. S. ABOUNAAJ
Copyright © 2013 SciRes.
74
ences; 4) information technology or mathematics; and 5) the
social or behavioural sciences. The final consideration is to
accommodate local and regional market needs with regard to
specific knowledge and skills required through appropriate
courses such as providing optional courses in areas relating to
networking and databases.
The learning outcomes of the Computer Science program and
their mapping to program educational objectives are shown in
Table 1.
Mapping Courses to Program Learning Outcomes
Courses were mapped to program learning outcomes by ex-
amining the individual learning outcomes of each course. When
one or more course learning outcome contributes to a particular
program learning outcome, then that course is linked to the
particular program outcome. Table 2 shows the mapping of
courses to program learning outcomes.
Assessment Tools, Measurement Tools, and Success
Criteria
To measure the achievements of each of the learning out-
comes, appropriate assessment tools, measurement tools, and
corresponding success criteria were developed as described in
Table 3. Each program learning outcome is assessed using one
or more of the assessment tools described in Table 3 as indi-
cated by Table 4.
Course Assessment Measure ment To ols
The following tools which are used to measure the achieve-
ments of course learning outcomes are given in Table 5. Table
6 facilitates the measurement of the level of achievement of
each learning outcome of a particular course. For each course
learning outcome I and tool J, the maximum students grade
and theiraverage scored gradeare entered. Learning outcome I
is achieved if the overall score is 70. Overall score is calcu-
lated as total scored/total_max where,
total_scored is the sum of average grades obtained by stu-
dents from all tools J for learning outcome I, and
total_max is the sum of the maximum grade of all tools J
for the learning outcome I.
Assessment Time Frame
The assessment and evaluation activities are conducted ac-
cording to the time frame shown in Table 7.
Data Collection
The following forms were designed to collect the required
assessment data:
1) Course Learning Outcomes Achievements Form: this form
is filled by course lecturer. The form contains information
based on calculations done in Table 6 showing the overall
score for each learning outcome of a particular course. De-
pending on the number of course learning outcomes which has
been achieved, a decision is made on whether the whole course
learning outcomes were achieved or not.
2) Exit Survey Form: this survey form contains questions re-
lating directly to the program learning outcomes.
3) Alumni Survey Form: this survey form contains questions
relating directly to the program learning outcomes.
4) Employer Survey Form: this survey form contains ques-
tions about program learning outcomes that can be evaluated
from an employer’s point of view.
5) Internship Survey Form: this survey form contains ques-
tions about internship learning outcomes which are directly
related to some of the program learning outcomes that can be
evaluated by the field supervisor of the trainee student.
Analysis of Program Learning Outcomes
The analysis starts by investigating the achievements of the
learning outcomes of each course mapped to program learning
outcomes. Then, each program learning outcome will be ana-
lysed individually in terms of the tools used to measure the
achievements of that learning outcome stated in Table 4.
Analysis of course learning outcomes
Based on the computations described in Table 6, a decision
has been reached regarding whether the course learning out-
comes have been achieved for each course as shown in Table 8.
The next step uses mapped courses to decide whether each
program learning outcome has been achieved. The percentages
of courses which have successfully contributed to the achieve-
ment of each program learning outcomes given in Table 9. It is
Table 1.
Mapping program educational objectives to program learning outcomes.
Objective After graduation, students should have:
1. An understanding of the theoretical foundations of computer science.
2. Analytical and critical thinking ability for problem solving.
3. An understanding of the principles of efficient program design techniques and strategies.
Objective#1
4. Knowledge, skills and use of a variety of systems and application software; hardware; computational algorithms; programming
languages; and human computer interfaces techniques.
5. Knowledge and skills to store, retrieve, and manipulate information.
6. The ability to analyze, design, implement, test, and evaluate a computer-based system.
7. The ability to work both independently and as team members.
8. The ability to communicate effectively orally and in writing.
Objective#2
9. An awareness of the ethical issues affecting computer science and the impact of computers on society.
Objective#3 10. The ability to pursue postgraduate study and research.
R. A. K. MEHDI, M. S. ABOUNAAJ
Table 2.
Mapping courses to program learning outcomes.
Program learning outcomes
No Course ID 1 2 3 4 5 6 7 8 9 10
1 110110 X
2 120110 X
3 310112 X X X
4 310211 X X X
5 310314 X
6 311221 X X
7 311223 X X X X
8 311242 X
9 311284 X
10 311300 X X X X
11 311311 X X X
12 311319 X X
13 311321 X X X
14 311323 X X X
15 311332 X X X X
16 311335 X X X X
17 311336 X X X
18 311342 X X
19 311422 X X X
20 311431 X X X X X X
21 311435 X X
22 311442 X X X X
23 311451 X X
24 311452 X X
25 311463 X X X X
26 311471 X X X
27 311472 X X X X X
28 312245 X X X
29 312381 X X
30 312382 X X
31 314241 X X X
32 400291 X X X
clear from Table 9 that program learning outcomes 1 and 2
were not achieved according to the criteria specified in Table 3.
Analysis of Exit and Alumni Survey Forms
Analysis of data collected from Exit and Alumni Survey
Forms is presented in Table 10. Results show that graduates are
satisfied with all program learning outcomes. The average rat-
ing score is the average of all the scores of the learning out-
comes based on all exit and alumni forms. The numbers in
bracket are the standard deviations.
Analysis of Employer Survey Forms
Data obtained from collected Employer Survey Forms were
analysed, and summary of the statistics is presented in Table 11.
The results indicate that employers are satisfied with the stan-
dard of the graduates, yet they believe that more efforts are
Copyright © 2013 SciRes. 75
R. A. K. MEHDI, M. S. ABOUNAAJ
Table 3.
Assessment tools, measurement tools, and success criteria.
# Assessment tools Measurement tool Success criteria
1. Courses mapped to a given program learning outcome. Course Performance Tools Percentage of related courses which have
achieved the learning outcome is 70%.
2.
Alumni Survey Forms: These forms ask the graduate to rate the
achievement of a given learning outcome on a scale of 1 to 5:1
being poor and 5 being outstanding.
Average rating obtained for
the learning outcome.
The average score for the learning outcome
from all Alumni Survey forms is 3.5
3.
Exit Survey Forms: These forms ask the graduate to rate the
achievement of a given learning outcome on a scale of 1 to 5:1 being
poor and 5 being outstanding.
The average score for the learning outcome
from all Exit Survey forms is 3.5
4.
Employer survey forms: These forms ask the graduate to rate the
achievement of a given learning outcome on a scale of 1 to 5:1 being
poor and 5 being outstanding.
The average score obtained from all
Employer Survey forms is 3.5
5.
Internship survey forms. This form asks the field supervisor to rate the
achievement of specific learning outcomes on a scale of 1 to 5:1 being
poor and 5 being outstanding.
6. Capstone project Percentage of students who
score grade B or above.
Percentage of students who score grade B or
above in capstone project is 70%.
Table 4.
Assessment tools used for each program learning outcome.
Program Learning outcome Assessment tools used Program learning outcome Assessment tools used
1 1, 2, 3 6 1, 4, 6
2 1, 4 7 4, 5, 6
3 1, 2, 3 8 1, 4, 5, 6
4 1, 4, 5 9 1, 4, 5
5 1, 4 10 1, 2, 3
Table 5.
Assessment tools used for each program learning outcome.
1. Student portfolio 2. Mid term exam 3. Final written exam
4. Short papers 5.Team projects 6. Oral discussion
7. Presentations 8. Tests & quizzes 9. Individualized products
10. Assignments 11. Lab work 12. Other
Table 6.
Calculating the achievements of course learning outcomes.
Tool ID: Tool ID: Tool ID:Tool ID:Tool ID:Tool ID:Tool ID:Tool ID:Tool ID: Tool ID: Total Overall scor e
Max grade
Learning
outcome #1 Scored grade
(average)
Max grade
Learning
outcome #2 Scored grade
(average)
Learning
outcome #··· ···
Max grade
Learning
outcome #n Scored grade
(average)
Copyright © 2013 SciRes.
76
R. A. K. MEHDI, M. S. ABOUNAAJ
Table 7.
Assessment time frame.
No Assessment tool Time frame
1 Course learning outcomes achievements form Start of each academic year.
2 Exit survey forms
3 Alumni survey forms
4 Employer survey forms
Assessment data collection is an on-going process; evaluation is conducted
once every four years (one year prior to reaccreditation).
5 Capstone project Start of each academic year.
6 Internship survey forms Start of each academic year.
Table 8.
Achievement oflearning outcomes for each individual course.
No Course ID Learning outcomes achievement (Yes/No)No Course ID Learning outcomes achievement (Yes/No)
1 110110 NO 17 311342 NO
2 120110 NO 18 311422 NO
3 310112 NO 19 311431 YES
4 310211 NO 20 311432 YES
5 310314 YES 21 311435 YES
6 311221 NO 22 311442 YES
7 311223 YES 23 311451 YES
8 311242 NO 24 311452 YES
9 311284 YES 25 311463 YES
10 311300 YES 26 311471 YES
11 311311 YES 27 311472 YES
12 311319 YES 28 312245 NO
13 311321 YES 29 312381 YES
14 311323 YES 30 312382 YES
15 311332 YES 31 314241 YES
16 311335 YES 32 400291 YES
required to produce graduates in line with the intended educa-
tional objectives of the program. The number in bracket is the
standard deviation.
Analysis of Internship Survey Forms
Internship Survey Forms collected were analysed, and a
summary of the statistics is given in Table 12. The number in
bracket is the standard deviation.
Achievement of Program Learning Outcomes
A decision on each program learning outcome is made by
combining results from different tools for that outcome. A pro-
gram learning outcome must pass all tools to be declared
achieved. Table 13 presents the final decision on the achieve-
ment of each program learning outcome.
Program Evaluation—Closing the Loop
Table 13 shows that the Computer Science Program on
which the above analysis was based needs to address each un-
derachieved or marginally achieved program learning outcomes,
and consider one or more of the remedial actions for each of
these program learning outcome. Program learning outcomes 1
and 2 must be readdressed, and measures need to be considered
to ensure that these learning outcomes are achieved satisfacto-
rily in the future.
The College Assessment Committee (CAC) has developed a
set of remedial actions to be considered in improving the level
of attainment of unsatisfactorily achieved program learning
outcomes. The remedial actions to be taken for unsatisfactory
program outcome depend on the assessment tool which has
caused that the program outcome to fail. The set of remedial
actions is listed below. Other appropriate actions may be con-
sidered as well.
1) Adding new knowledge units to a course.
2) Refining or deleting certain course knowledge units.
3) Changing prerequisite courses.
4) Introducing appropriate elective courses.
5) Increasing the number of or changing the nature of course
Copyright © 2013 SciRes. 77
R. A. K. MEHDI, M. S. ABOUNAAJ
Table 9.
Percentage of courses that contributed to each program learning outcome.
Program learning outcomes
No Course ID 1 2 3 4 5 6 7 8 9 10
1 110110 No
2 120110 No
3 310112 No No No
4 310211 No No No
5 310314 Yes
6 311221 No No
7 311223 Yes Yes Yes Yes
8 311242 No
9 311284 Yes
10 311300 Yes Yes Yes Yes
11 311311 Yes Yes Yes
12 311319 Yes Yes
13 311321 Yes Yes Yes
14 311323 Yes Yes Yes
15 311332 Yes Yes Yes Yes
16 311335 Yes Yes Yes Yes
17 311336 N/A N/A N/A
18 311342 No No
19 311422 No No No
20 311431 Yes Yes Yes Yes Yes Yes
21 311435 Yes Yes
22 311442 Yes Yes Yes Yes
23 311451 Yes Yes
24 311452 Yes Yes
25 311463 Yes Yes Yes Yes
26 311471 Yes Yes Yes
27 311472 Yes Yes Yes Yes Yes
28 312245 No No No
29 312381 Yes Yes
30 312382 N/A N/A
31 314241 Yes Yes Yes
32 400291 Yes Yes Yes
Percentage 56% 54% 83% 71% 100% 81% 100% 100% 100% 100%
Table 10.
Analysis of exist and alumni survey forms.
Program Learning outcome# Program learning outcome Summary statistics
1 An understanding of the theoretical foundations of computer science. 4.13 (0.81)
2 Analytical and critical thinking ability for problem solving. 3.73 (0.86)
3 An understanding of the principles of efficient program design techniques and strategies. 3.82 (0.97)
4 Knowledge, skills and use of variety of system and application software, hardware, computational
algorithms, programming languages and human computer interfaces techniques. 3.58 (1.0)
5 Knowledge and skills to store, retrieve and manipulate information. 3.83 (0.94)
6 The ability to analyse, design, implements, test, and evaluate a computer-based system. 3.78 (0.87)
7 The ability to work both independently and as team members. 4.17 (0.91)
8 The ability to communicate effectively orally and in writing. 3.87 (0.89)
9 An awareness of the ethical issues affecting computer science and the impact of computers on society. 3.62 (1.04)
10 The ability to pursue postgraduate study and research. 3.65 (1.10)
Copyright © 2013 SciRes.
78
R. A. K. MEHDI, M. S. ABOUNAAJ
Table 11.
Scores obtained for program learning outcomes from employer survey forms.
Program learning outcome# Program learning outcome Summary statistics
2 Analytical and critical thinking ability for problem solving. 3.91 (0.75)
4 Knowledge, skills and use of variety of system and application software, hardware, computational algorithms,
programming languages and human computer interfaces techniques. 3.82 (0.80)
5 Knowledge and skills to store, retrieve, and manipulate information. 3.39 (0.61)
6 The ability to analyse, design, implements, tests, and evaluate a computer-based system. 3.45 (0.89)
7 The ability to work both independently and as team members. 3.90 (0.72)
8 The ability to communicate effectively orally and in writing. 3.55 (0.89)
9 An awareness of the ethical issues affecting computer science and the impact of computers on society. 3.77 (0.81)
Table 12.
Scores obtained for program learning outcomes from internship survey forms.
Program Learning outcome# Program learning outcome Summary statistics
4 Knowledge, skills and use of variety of system and application software, hardware, computational
algorithms, programming languages and human computer interfaces techniques. 1.29 (0.77)
7 The ability to work both independently and as team members. 4.03 (0.87)
8 The ability to communicate effectively orally and in writing. 4.56 (0.56)
9 An awareness of the ethical issues affecting computer science and the impact of computers on society. 4.59 (0.62)
Table 13.
Program learning outcomes achievements.
Learning outcome# Learning outcome Achievement (Yes/No)
1 An understanding of the theoretical foundations of computer science. No
2 Analytical and critical thinking ability for problem solving. No
3 An understanding of the principles of efficient program design techniques and strategies. Yes
4 Knowledge, skills and use of variety of system and application software, hardware, computational
algorithms, programming languages and human computer interfaces techniques. Yes
5 Knowledge and skills to store, retrieve and manipulate information. Yes
6 The ability to analyse, design, implement, test, and evaluate a computer-based system. Yes
7 The ability to work both independently and as team members. Yes
8 The ability to communicate effectively orally and in writing. Yes
9 An awareness of the ethical issues affecting computer science and the impact of computers on society. Yes
10 The ability to pursue postgraduate study and research. Yes
Percentage of program learning outcomes achieved 80%
assignments.
6) Revising the practical components of mapped courses.
7) Giving more emphasis to independent work done by stu-
dents in courses relating to a program learning outcome.
8) Changing textbook or course references.
9) Changing course delivery methods.
10) Providing support structures such as tutoring or help ses-
sions.
11) Refining or changing the learning outcome.
12) Refining evaluation methods.
13) Refining implementation of the assessment process
14) Refining criteria used in the evaluation.
15) Changing course instructor.
16) Provide professional development program for faculty in
learning outcomes and assessment.
17) Recommending additional research, assessment, and
evaluation in case of unclear evidence
Figure 2 describes how the results of the assessment process
are used in improving the curriculum within the guidelines of
the Ministry of Higher Education-Commission for Academic
Accreditation standards and the ACM curriculum. Information
regarding lecturer course feedback and student course feedback
will be obtained from two survey forms designed for this pur-
pose. Students fillin Course Evaluation Form online each se-
mester and data is accessed by members of the College As-
sessment Committee. The Lecturer Course Assessment form is
Copyright © 2013 SciRes. 79
R. A. K. MEHDI, M. S. ABOUNAAJ
Figure 2.
Program assessment process.
filled in by each faculty member at the end of the semester for
each course taught by the faculty member in that semester.
Conclusion and Future Work
This paper describes a model that the College of Information
Technology at Ajman University of Science and Technology
uses to implement a complete outcome-based assessment, and
evaluation process for the accreditation of its programs. The
procedures and tools described were capable of showing the
degree to which each program learning outcome has been
achieved. The model is generic in the sense that it can be ap-
plied to any computing program with measurable learning out-
comes. Each academic program has to identify its own set of
actions to deal with each underachieved program learning out-
come. Implementing a systematic assessment and quality as-
surance process model will help academic institutions to iden-
tify problematic areas and take the appropriate remedial actions.
The assessment scheme described in this document is continu-
ously updated as new measurement tools are developed or pro-
gram learning outcomes revised. The model described assumes
that courses contributing to a program learning outcome have
an equal weight on that learning outcome; however, this may
not be accurate, and different weights may have to be used to
reflect the contribution of each course to a particular program
learning outcome. Different thresholds to measure the achieve-
ments of course learning outcomes might have to be used for
different courses. For example, courses that are used to filter
students, such as mathematics and programming, early in the
program may require lower thresholds than other courses. In
this study, the same threshold (70% of students achieve grade C
or above) was used for all courses. This point will be given due
consideration in the incoming assessment activity. Currently,
data collection and analysis are donemanually. Faculty mem-
bers have to devote a considerable amount of time to these
tasks. Computerizing the process will reduce much of the bur-
den currently endured by faculty members.
REFERENCES
ABET Computing Accreditation Commission (2010). Criteria for ac-
crediting computing programs. URL (last checked 14 March 2009).
http://www.abet.org
Adam, S. (2004). A consideration of the nature, role, application and
implications for European education of employin g learning outcomes
at the local, national and international levels. Edinburgh: United
Kingdom Bologna Seminar, Heriot Watt University.
Angelo, T. A. (1995). Reassessing and defining assessment. American
Association for Higher Education Bulletin, 48, 7-9.
Ashiem, C., Gowan, A. & Reichgelt, H. (2007). Establishing an as-
sessment process for a computing program. Information Systems
Education Journal, 5, 1-12.
Ashiem, C., Gowan, A., O’Malley, J., & Reichgel, H., (2007). An IT
program assessment process. 8th Conference on Information Tech-
nology Education, SIGITE 2007, Destin, 18-20 October 2007.
Association of Computing Machinery (ACM), & IEEE Computer Soci-
ety (2008). Computer Science Curriculum 2008: An Interim Revision
of CS 2001. URL (last checked 16 June 2011).
http://www.acm.org/education/curricula/ComputerScience2008.pdf
Bailie, F., Marion, B., & Whitfield, D. (2010). How rubrics that meas-
ure outcomes can complete the assessment loop. Journal of Comput-
ing Sciences in Colleges, 2 5, 15-28.
Blanford, D., & Hwang, D. (2003). Five easy but effective assessment
methods. SIGCSE’03 Proceedings of the 34th SIGCSE technical
symposium on Computer science education, 35, 41-44.
Buzzetto-More, N., & Alade, A. (2006). Best practices in e-assessment.
Journal of Information Technology Education, 5, 251-269.
Crouch, B., & Schwartzman, L. (2003). Computer science accreditation:
the advantages of being different. ACM SIGCSE Bulletin, 35, 36-40.
doi:10.1145/792548.611927
Gowan, A., MacDonald, B., & Reichgelt, H. (2006). A configurable
Copyright © 2013 SciRes.
80
R. A. K. MEHDI, M. S. ABOUNAAJ
assessment information system. The 7th Conference on Information
Technology Education, SIG ITE 2006, Minneapolis, October 19-21.
Jones, L., & Price, A. (2002). Changes in computer science accredita-
tion. Communications of the ACM, 45, 99-103.
doi:10.1145/545151.545158
Kellough, R. D., & Kellough, N. G. (1999). Seconda ry sc h o ol teaching:
A guide to method and resources planning for competence. Upper
Saddle River, New Jersey: Prentice Hall.
Kohun, F., & Wood, D. (2003). The ABET CAC accreditation experi-
ence—intent and reality—the information systems perspectives. In-
formation Systems Education Journal, 1, 1-11.
Love, T., & Cooper, T. (2004). Designining online information systems
for portfolio-based assessment: Design criteria and heuristics. Jour-
nal of Information Techn o l o g y Education, 3, 65-81.
Maxim, B. (2004). Closing the loop: Assessment and accreditation,
Journal of Computing Sciences in Colleges, 20, 7-18.
Sanders, K., & McCartney, R. (2003). Program assessment tools in
computer science: A report from the trenches, ACM SIGCSE Bulletin,
35, 31-35. doi:10.1145/792548.611926
Copyright © 2013 SciRes. 81