DIGITAL MEDIA INSTITUTEhypermedia laboratory

Finnish Virtual UniversityTAMPERE UNIVERSITY OF TECHNOLOGY

A Multidisciplinary tool for the evaluation of usability, pedagogical usability, accessibility and

informational quality of web-based courses

Kirsi Silius, Anne-Maritta Tervakari &

Seppo Pohjolainen

Tampere University of Technology (TUT) Finland

PEG2003 St.Petersburg, Russian

2DIGITAL MEDIA INSTITUTEhypermedia laboratory

Finnish Virtual UniversityTAMPERE UNIVERSITY OF TECHNOLOGY

The main phases of survey at TUT

– Context analysis: to find significant background factors, which have strong impact to the instructional value of the certain web-based learning environment.

– Advantages and disadvantages of web-based learning in various contexts. What are the critical factors of quality in web-based learning?

– Assessment of implementation of web courses using multidisciplinary framework based on usability, pedagogical usability, accessibility and informational quality

– Value added of web-based courses in various contexts.

3DIGITAL MEDIA INSTITUTEhypermedia laboratory

Finnish Virtual UniversityTAMPERE UNIVERSITY OF TECHNOLOGY

Viewpoints of evaluation

usabilityusability

accessibilityaccessibility

presentation of information

presentation of information

reliability of information

reliability of information

pedagogical usability

pedagogical usability

value addedvalue added

user group user group

user group user group

quality of information

fluenc

y of

use

instructional quality be

nefit

s

4DIGITAL MEDIA INSTITUTEhypermedia laboratory

Finnish Virtual UniversityTAMPERE UNIVERSITY OF TECHNOLOGY

Usefulness of web-based learning environments

based on Nielsen (1993), modified by Tervakari & Silius 2002-03

Utility

Easy to learn

Efficient to use

Usability

Easy to remember

Few errorsSubjectively pleasing

Value added

Support for organisation of the teaching process

Support for learning, tutoring and scaffolding process and support for the achievement of learning objectives

Support for the development of learning skills (interaction with other actors and growth of learners` autonomy

Organisation of teaching process

Development of quality of teaching

Development of learners’ learning skills

Testing and development of educational ICT

Pedagogical usability

Usefulness

Social acceptability

Sy

ste

m a

cc

ep

tab

ilit

y

Practical acceptability

Cost

CompatibilityReliability

Etc.

Nielsen, J. 1993. Usability Engineerings. Boston (MS.): Academic press.

5DIGITAL MEDIA INSTITUTEhypermedia laboratory

Finnish Virtual UniversityTAMPERE UNIVERSITY OF TECHNOLOGY

Evaluation sections

6DIGITAL MEDIA INSTITUTEhypermedia laboratory

Finnish Virtual UniversityTAMPERE UNIVERSITY OF TECHNOLOGY

Purpose of this evaluation tool

• This tool is not – for making ranking lists of web-based courses– for evaluate the validity or substantiveness of the

content

• This tool is to – help teachers to pay attention to properties of

their web pages which might need some improvement

7DIGITAL MEDIA INSTITUTEhypermedia laboratory

Finnish Virtual UniversityTAMPERE UNIVERSITY OF TECHNOLOGY

Start of evaluation

1.

2.

3.

8DIGITAL MEDIA INSTITUTEhypermedia laboratory

Finnish Virtual UniversityTAMPERE UNIVERSITY OF TECHNOLOGY

Choose sections

9DIGITAL MEDIA INSTITUTEhypermedia laboratory

Finnish Virtual UniversityTAMPERE UNIVERSITY OF TECHNOLOGY

Questions to be answered

10DIGITAL MEDIA INSTITUTEhypermedia laboratory

Finnish Virtual UniversityTAMPERE UNIVERSITY OF TECHNOLOGY

Report 1: Results of the evaluation

11DIGITAL MEDIA INSTITUTEhypermedia laboratory

Finnish Virtual UniversityTAMPERE UNIVERSITY OF TECHNOLOGY

Report 2: Guides and tips

12DIGITAL MEDIA INSTITUTEhypermedia laboratory

Finnish Virtual UniversityTAMPERE UNIVERSITY OF TECHNOLOGY

Report 3: Own answers

16DIGITAL MEDIA INSTITUTEhypermedia laboratory

Finnish Virtual UniversityTAMPERE UNIVERSITY OF TECHNOLOGY

Futher development

• More criteria and guidelines into pedagogical usability section

• More references into all sections• Questionnaires for real users• The tool will help to match data

collected from evaluators and real users