children: can they inspect it? yes they can! gavin sim
TRANSCRIPT
Children: Can they inspect it? Yes they can!
Gavin Sim
Overview
• Introduce Inspection Methods• Known Issues with Adults• Details of heuristic evaluation with
Children• Discussion of results• Conclusions and Future Work
Inspection Methods
• Informal methods that rely on the judgement of the evaluator to predict problems
• Walkthrough– Need understanding of cognitive processes,
interface and task list and expected actions. expert evaluator
• Heuristic evaluation– Need set of appropriate heuristics, application, 3 –
5 expert evaluators, individually find problems, merge form single list
Why 3-5 is enough?
(Jakob Nielsen & Landauer, 1993) claim that a typical value
of λ to be 31% Jakob Nielsen and Thomas K. Landauer. 1993. A mathematical model of the finding of usability problems. In Proceedings of the
INTERACT '93 and CHI '93 conference on Human factors in computing systems (CHI '93). ACM, New York, NY, USA, 206-213.
Known Issues
• Double Experts – experts in system under investigation and usability
• Evaluator Effect – find different numbers of problems
• Predict problems that are not real – can waste time and resources in development
Children and Heuristic Evaluations
• 14 children between the ages of 10-11 years participated in this study.
• They were in four different groups – x3 groups of 3– 1 group of 2
• 3 researchers were recruited– 1 as facilitator – 2 as observers.
Method
• Touchscreen laptop with a music making game called “JamMo”.
• The heuristic set by Korhonen and Koivisto was used which specifically related to games
• Modified to simplify the terminologyKorhonen, H. and E.M. Koivisto. Playability Heuristics for Mobile Games. in MobileHCI. 2006. Helsinki: ACM
Method
• Children – Given forms to capture problems – Set of heuristics and severity scales
• Adult Observers– Given forms to record issues in the method
Results
• Could children preform a heuristic evaluation?– All the children successfully conduct a heuristic
evaluation. – Overall 27 usability problems were found with an
average of 1.71 problems per participant across all the groups.
• Genuine problems reported:– Drag and drop functionality was the most recorded
issues with 6 out of 14 participants noting this – Confusion about using a cupboard to save work was
mentioned.
Observed Problems
Indicators Total Problems Just exploring not evaluating 18
Got lost and did not complete 13
Really Stuck -beyond them 4
Did not follow procedures 5
Had difficulty understanding heuristics 23
Had difficulty understanding severity 13
Technical Issues 3
Suggested solution 2
Data Capture Forms
• There were issues with the children being able to record the data on the forms
• Children struggled with too many pieces of paper
• First form did not work well – re-designed
Conclusions
• The study confirmed that children can critically evaluate a product and can identify genuine problems.
• Communication skills is still a challenge• Careful design of forms
Further Research
• Need to design an appropriate severity rating scale – bad scale might work
• Age appropriate heuristics are needed
• Stopping them just playing• PhD student looking at other
inspection methods
