april 2004iadr honolulu nikos mattheos nikos mattheos malmö university, sweden interactive software...
TRANSCRIPT
IADR Honolulu Nikos MattheosApril 2004
Nikos MattheosNikos MattheosMalmö University, SwedenMalmö University, Sweden
Interactive softwareInteractive software and self-assessment skillsand self-assessment skills
April 2004 IADR Honolulu Nikos Mattheos
Self assessment ability:Self assessment ability:
•Evaluate professional actions – process /outcomeEvaluate professional actions – process /outcome
• Compare to standardsCompare to standards
• Identify weaknessesIdentify weaknesses
• Define new learning objectivesDefine new learning objectives
April 2004 IADR Honolulu Nikos Mattheos
self assessment ability self assessment ability
reflectionreflection feedbackfeedback
April 2004 IADR Honolulu Nikos Mattheos
The Leo:The Leo:A free Internet-based platform for assessing and A free Internet-based platform for assessing and
training of self-assessment skills training of self-assessment skills in oral health education in oral health education (A. Nattestad – R. Attström)(A. Nattestad – R. Attström)
http://tmk.odont.ku.dk/leo/http://tmk.odont.ku.dk/leo/
April 2004 IADR Honolulu Nikos Mattheos
April 2004 IADR Honolulu Nikos Mattheos
April 2004 IADR Honolulu Nikos Mattheos
April 2004 IADR Honolulu Nikos Mattheos
Student’s essayStudent’s essay
Primary feedback: Primary feedback: Expert’s essayExpert’s essay
reflectionreflection
improvementimprovement?
April 2004 IADR Honolulu Nikos Mattheos
LeoLeon=26n=26
StaticStaticn=26n=26
BBaasseelliinnee
week 1 week 2 week 3 week 4 week 5
AAsssseessssmmeenntt
CaseCase
11
CaseCase
11 1
1CaseCase
22CaseCase
33CaseCase
44
CaseCase
22CaseCase
33CaseCase
442
2 3
3
4
4
April 2004 IADR Honolulu Nikos Mattheos
BBaasseelliinnee
• background data, demographicsbackground data, demographics
• competence with computers (score 1-50)competence with computers (score 1-50)
Baseline measurements: Baseline measurements:
April 2004 IADR Honolulu Nikos Mattheos
Weekly evaluations: Weekly evaluations:
•use of the software use of the software
• attitudes, acceptance, expectationsattitudes, acceptance, expectations (Visual Analogue Scale 1-16)(Visual Analogue Scale 1-16)
1
1 2
2 3
3 4
4
April 2004 IADR Honolulu Nikos Mattheos
AAsssseessssmmeenntt
The Interactive Examination:The Interactive Examination:
• self assessment as staring point (Internet)self assessment as staring point (Internet)
• oral + essay + written comparison with expertoral + essay + written comparison with expert
Mattheos N, Nattestad A, Falk Nilsson E, Attström R. The Interactive Examination: Assessing students’ self-assessment ability. Med Educ 2004, in press.
April 2004 IADR Honolulu Nikos Mattheos
ResultsResults
• Improved performance in general Improved performance in general
• No significant difference between groupsNo significant difference between groups (moderate agreement between 2 assessors)(moderate agreement between 2 assessors)
• Time on task correlates to written performanceTime on task correlates to written performance
• Computer competence correlates to written performanceComputer competence correlates to written performance
April 2004 IADR Honolulu Nikos Mattheos
• students judged 3 out of 4 cases as “easy”students judged 3 out of 4 cases as “easy”
• students were positive on Internet cases, students were positive on Internet cases, but some complained about workload but some complained about workload
• Attitude swiftsAttitude swifts
ResultsResults
April 2004 IADR Honolulu Nikos Mattheos
1. What do you think of the Internet application you used? (VAS 1-16) Leo group Non Leo group difference
meanafter case 1
10,15
8,3
p>0,05
meanafter case 4
10,5 10,8 p>0,05
difference p>0,05 p = 0,0148
2. How much do you think that the software you used will help you at the exams?
Leo group Non Leo group difference
meanafter case 1
11,2 8,8 p = 0,0091
meanafter case 4
10,9 9,9 p>0,05
difference p>0,05 p>0,05
3. How much have you learned through the Internet cases?
Leo group Non Leo group difference
meanafter case 1
6,7 5,43 p>0,05
meanafter case 4
10 9,43 p>0,05
difference p = 0,0032 p = 0,0004
April 2004 IADR Honolulu Nikos Mattheos
1. What do you think of the Internet application you used? (VAS 1-16) Leo group Non Leo group difference
meanafter case 1
10,15
8,3
p>0,05
meanafter case 4
10,5 10,8 p>0,05
difference p>0,05 p = 0,0148
2. How much do you think that the software you used will help you at the exams?
Leo group Non Leo group difference
meanafter case 1
11,2 8,8 p = 0,0091
meanafter case 4
10,9 9,9 p>0,05
difference p>0,05 p>0,05
3. How much have you learned through the Internet cases?
Leo group Non Leo group difference
meanafter case 1
6,7 5,43 p>0,05
meanafter case 4
10 9,43 p>0,05
difference p = 0,0032 p = 0,0004
April 2004 IADR Honolulu Nikos Mattheos
1. What do you think of the Internet application you used? (VAS 1-16) Leo group Non Leo group difference
meanafter case 1
10,15
8,3
p>0,05
meanafter case 4
10,5 10,8 p>0,05
difference p>0,05 p = 0,0148
2. How much do you think that the software you used will help you at the exams?
Leo group Non Leo group difference
meanafter case 1
11,2 8,8 p = 0,0091
meanafter case 4
10,9 9,9 p>0,05
difference p>0,05 p>0,05
3. How much have you learned through the Internet cases?
Leo group Non Leo group difference
meanafter case 1
6,7 5,43 p>0,05
meanafter case 4
10 9,43 p>0,05
difference p = 0,0032 p = 0,0004
April 2004 IADR Honolulu Nikos Mattheos
Reflections…Reflections…
• Human feedback irreplaceable?Human feedback irreplaceable?• Content and cases too easy?Content and cases too easy?• Time not enough? Time not enough? •Assessment procedure not sensitive enough ?Assessment procedure not sensitive enough ?
