Assessment and EvaluationBS-CSE Program

Neelam SoundarajanChair

CSE Undergrad Studies Comm.

Objectives, Outcomes

PEOs (fairly standard): ... related to employment in

profession ... ... related to advanced studies ... ... related to being informed/involved

members of community and profession ...

Student Outcomes (fairly standard): (a) - (k) (slightly modified) (l) - (n) (as required by CAC Criteria)

Assessment Methods

Alumni Surveys (next round?) Advisory Board (annual) Exit Survey Annual Student Forum POCAT (Exit test) Rubrics in Capstone CoursesAll of the above are standard except for POCAT (Program OutComes Achievement Test)

Assessment Methods

Slides on Alum Surveys etc. to be created ...

POCAT

All students are required to take it; Typically in the graduation

semester; Assesses the program, not the

students; Each test paper has a unique code

on it; no record of (student code) mapping!

Only the student knows his/her code;

So: when results are posted, only the student can know how he/she did;

Goal: Assess the program, not the students;

POCAT (contd)

20+ multiple-choice questions; Topics: Reqd. courses, popular

electives; Conceptual questions designed to

tease out common misunderstandings;

Maybe "cross-cutting" (across courses);

Questions are freely reused! No "security" concerns; Students never cheat ... why would

they?!

POCAT: Example Question

Your sibling is very ill and needs medication that you cannot afford, so you steal it. Which of the following is the ethical question that arises in this scenario:a. What kind of illness does your sibling have and

how do we prevent its spread among populations?

b. Why is the medication not affordable and how do we make it more affordable?

c. Is it ever right to steal, even if you have a great need?

d. What consequences might you face if you are caught?

e. All of the above are ethical questions.f. All of the above are important questions but

none is an ethical question.g. I cannot decide based on the given information.

POCAT Results

Note: Correct answers are in boldface

POCAT Results (contd)

• Shown: No. of students who picked each answer (bold=correct)• Nos. in () ("good") and [] ("bad") are important;

"good": measures of how many top students picked correct ans/ did not pick wrong answer;"bad": measures of how many top students picked wrong ans/ did not pick correct answer;

• For ethics question, (e) is a good distractor.

POCAT Results (contd)

Another example: In a question related to databases, students were often picking one particular wrong answer in preference to another wrong ans. although both looked equally plausible ...

We couldn't fix the problem (by modifying the course content) without knowing why!

One of the student reps on the Undergrad Comm. came up with a plausible explanation ...

POCAT Results (contd)

So: we have revised the question to test his hypothesis and plan to use it in the next POCAT;

If it turns out right, we will know what changes are needed in the DB course to fix the problem;

Summary

That is the essence of assessment, evaluation, and program improvement: Evaluate the results of the

assessments to help identify specific problems in the program and make changes in the program to address the problems.

Conclusion: The approach works well and is sustainable!