summary of study use kirkpatrik

Study Title Evaluating an Intelligent Tutoring System for Design

Patterns: the DEPTHSExperience.

Author

Zoran Jeremić1, Jelena Jovanović1 and Dragan Gašević2

1FON – School of Business Administration, University of Belgrade // jeremycod@yahoo.com // jeljov@gmail.com2School of Computing and Information Systems, Athabasca University, Canada // dgasevic@acm.org.

The purpose of the study

This paper describes the evaluation study of DEPTHS, an intelligent tutoring system for learning software design patterns, and aimed at assessing the system’s effectiveness and the accuracy of the applied student model. It also targeted the evaluation of the students’ subjective experiences with the system.

The audience:

software engineering students,. School of Business Administration, University

of Belgrade.

This study used Kirkpatrick Models:

1- Evaluation of reactions: by use an open-ended questionnaire.

2- Evaluation of learning: Endres and Kleiner (1990) state that pretests and posttests are necessary when evaluating the amount of learning that has taken place.

3- Evaluation of behavior: Feedback from students, their supervisors, and peers as well as some other techniques can be used for collecting information at this level.

4- Evaluation of results:

Students who learned with DEPTHS found that the system helped them to learn a lot about design patterns.They were especially pleased with the system’s ability to provide them with many useful information, feedbackmessages and advice for further work. Students’ responses indicated the need for regular communication withteachers and other students as the underpinning priorities for successful completion of online learning.