TURKU UNIVERSITY OF APPLIED SCIENCESBusiness Information Systems (MBA)

Master's Thesis Presentation

May 2014By

Youssef Hammouche

The rapid changes of information technology, present a new challenge and awareness of the necessity to adopt new tools and processes to improve the quality of education system.

In confronting this challenge it became obligatory to know the complexity of the education system from student’s perspective and related problems that should be addressed.

every education system must have an independent strategy of changes to determine the affecting factors of failure or success in education system.

The usability testing represent one of the success factors to be implemented into any LMS to evaluate the examined interfaces by testing it on users.

It became an essential component of LMS development process similar to software testing and system design.

LMS might be very difficult to use by students if usability problems are not identified in early stage

Many LMS projects have failed during last few years to stay among the top existing systems, reasons could be numerous but lack of usability testing is considered as one of the most common reasons.

especially that students interactions with system is highly impacted with various barriers: lack of available support by teachers, poorly motivated to understand system's features, students are unsure in the use of ICT.

Therefore it is mandatory to conduct usability testing during the system development cycle.

CONTENT

LIST OF ABBREVIATIONS (OR) SYMBOLS1. INTRODUCTION 1.1. Problem Statement and Research Objectives

1.2. Research Methodology 1.3. Optima LMS

2. LEARNING MANAGEMENT SYSTEMS LMSs 2.1. Open Source LMS

2.2. CLOUD LMS 2.3. Steps in Selecting a LMS 2.4. Key Factors of Successful LMS

3. THE USABILITY TESTING PLAN OF OPTIMA LMS 3.1. The Concept of Usability Testing 3.1.1. What is Usability? 3.1.2. What is Usability Testing 3.1.3. The Purpose of Usability Testing 3.1.4. What to Test?3.2. Usability Test Plan for Optima LMS

3.2.1. Problem Statement and Test Objectives 3.2.2. User Profile 3.2.3. Methodology 3.2.3.1. Participants 3.2.3.2. Length of Sessions 3.2.3.3. Quantitative & Qualitative Data 3.2.4. Scenarios and Tasks 3.2.5. Test environment and Equipment

4. EMPIRICAL USABILITY TESTING IN OPTIMA LMS 4.1. Data Collection

4.2. Test Results and Analysis 4.2.1. Top Negative Finding 4.2.1.1. Website Layout and Login Process 4.2.1.2. System Features and Editing Options 4.2.2. Top Positive Finding 4.2.3. Findings Log from the Observer 4.2.3.1. Task Completion Rate 4.2.3.2. Time on Task

4.2.3.3. Problems and Helps required

4.2.4. Presenting Results Based on Post-task and Post-test Questionnaires

5. USABILITY RECOMMENDATIONS FOR OPTIMA LMS DEVELOPERS

6. DISCUSSION REFERENCES

APPENDICESAppendix 1. Pre-test Questionnaire.

Appendix 2. Post-task Questionnaire. Appendix 3. Post-test Questionnaire. Appendix 4. Usability Testing Video Permission

Form

Discendum is the optima system developer; they consider it as a high-quality system with an easy tool to be used by users to build a perfect online environment.

Optima service aimed to become one of the best existing platforms for eLearning courses, information management, networking, and supporting learning on the job.

Actually, more than 200 educational institutions and companies have implemented Optima to support their education and projects.

Actually, more than 200 educational institutions and companies have implemented Optima to support their education and projects.

Based on the cited quality of Optima and growing interest of customers, the system is not yet classified among the top learning management systems.

Only a few academic literatures have been written under the name of Optima.

often those documents belong to libraries of universities which kept a right for only their users to access into it.

This limitation makes the case more complicated and stands as a barrier of sharing Optima researches to external readers of Finnish universities

Additionally, lack of usability testing researches has been conducted on the subject, which makes my research a bit difficult to collect necessary information and to reach my study goals.

On the other hand, similar learning management system has taken a wide attention across the world, despite that Optima has improved all interfaces by merging versatile tools, flexible online environment, and high security access "Moodle serve over 70 million users“

Do Optima system provide enough functionality to users to help them achieve their study goals?

Is the slow success of Optima due to the usability of system which would reflect the users’ satisfaction and dissatisfaction to the system?

Is Optima system afford enough information and help to users with less effort and less clicks of navigation?

Is Optima Easy –to- use?

In order to get responses to the previous questions, a usability testing should be taken into consideration to find out how real users use the system in real life, and which kind of difficulties and problems users might encounter during their daily tasks?

This thesis concentrates onto students interface only, to design a test plan based on the literature of Carol M. Barnum (Usability Testing Essentials Ready, Set…Test), and Joseph S. Dumas (A Practical Guide to Usability Testing), and other resources as well.

Constructive research, it is considered as one of the options currently available for a case researcher, and it has been widely developed in the field of business administration and IS.

It has gained significant results by implementing new construction to solve problems faced in real world, and this involves the analysis of the users and performance of the system design in order to understand, explain, and improve the designed systems.

This research method matched much on my study research, where I am supposed to understand Optima system environment, and try to find usability problems encountered by users.

The new construction issue in this research is to design a test plan based on theoretical part which frequently includes a test objectives, methods, and scenarios.

The practical part is reserved to apply the test plan and to collect data based on the selected methods, for a goal to be analysed, interpreted, and presented in form of recommendations to project stakeholders.

Optima is a web-based learning platform that supports learning environment and delivers extensive training programs, managing sales trainings, and offers opportunities to utilize different learning models and pedagogical ideas.

Optima is a Finnish company developed by Discendum together with its customers, company was established in 2001 for a goal to create and implement online learning managements and related training consultation services.

Company has delivered over 200 online learning solutions to various sorts of customers such as institution of higher education, companies, and other organizations.

The Company turnover of the fiscal year 2011 was approximately 1, 6 million euros; currently company employ 18 people in cities of Helsinki, Tampere, Rauma, and Oulu. Optima headquarters are located in Oulu.

What is Usability?ISO 9241: "The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” still wider: Specified users: it concerns the persons to whom the product is designed for. Specified goals: Are the shared goals between the users and product.

A specified context of use: Is the environment where the product should work by the users.

EX:“website, tools, mobile devices, software application, or anything designed for people.

Whitney Quesenbery' has given an easy definition to remember, called 5Es:

Effective:" How the user could complete the task and reach the goals?"

Efficient: "How fast the task can be done?"

Engaging: "Is the interaction between users and designed interface, and how satisfied are they about it?“

Error tolerant: "How many errors committed by users? How easily do users can recover from it?"

Easy to learn: "How easy could the user

accomplish the basic tasks for the first time they encounter a new product or service?" (Whitney Quesenbery. February 2004.)

What is Usability Testing? Represents all activities that focuses on

observing users testing a product or service in certain working conditions,

the observer is supposed to watch, listen and take notes while the participants are trying to complete tasks

for a goal to identify usability problems, collect qualitative and quantitative data, and to determine participants satisfaction and dissatisfaction with the product or service. Generally the usability testing has two forms:

Formative testing: while the product is still in the development process, for a goal to diagnose the problems and try to fix them before represented to the consumers.

Summative testing: commonly when the product is finished, with a goal to validate if the product meets the consumers requirements. (www.usability.gov).

The Purpose of Usability TestingUT became a necessary condition for a Website to survive.Web visitors are not trying seriously to spend much effort to understand what the Web offers for them as information or service, they leave the website as soon as they find difficulties to use it or to figure out what the web interface is trying to present for them. But the case might be totally different if you are forced to use a website or system which you are not familiar with and it has not been well tested.

therefore the usability testing should be adopted for any system to support users and to identify problems based on their experiences.

Reasons to test optima LMS: the number of conducted UT into Moodle

CMS is incomparable with any other LMS although Optima has almost the same

features as Moodle it has been used during a long period by our

school educators and students

it has gained a wide success by the users due to the free of charge of using the system by the open universities, university staff, and for the academic research purposes as well.

Other reason is to find out how students could easily use Optima platform for e-learning courses, support learning tools, information management, and as a space for interaction and collaboration between students and supervisors.

What to Test? Due to many reasons I have selected student as an interface to be tested during this usability testing to understand their experiences with the system and to determine whether the usability of it matches their expectations and support their goals.

Students are confronting more usability problems and difficulties compared to other users.

Easy to include students in testing process than supervisors.

number of students that are using Optima system are extremely higher compared to other interface users.

The common statement in usability testing is to inform the participants that "We are testing the product, not you".

Will document all the essential test planning steps and methods that will be used by me, to verify and ensure that Optima system meets the requirement of testing objectives.

will be presented as a formal report to record all the important activities related to the tested interface,

to figure out how the system would be tested, and to know the persons who will be recruited for this testing. The final report will be as a part of my final thesis work

The test plan will include the sections below:

Title page, Table of Contents, List of Illustration, Abbreviation & Figures Executive Summary, Problem Statement and

Test Objectives, Methodology, User Profiles Participant Incentive, Screeners, Scenarios &

Tasks, Evaluation Methods, Test environment & Equipment, Presentation of Findings,

Pre-test Questionnaire, Post-task & Post-test Questionnaire, Recommendations, & Appendices

is to discover whether Optima system is easy to be used and learned from students interface,

to find out if they could perform proposed tasks within a pre-determined time frame and how well they could understand it,

to ensures if the system design works as intended and meets users expectations.

To find serious usability problems which are related to system navigation, work functionality, message errors and system features.

Finally is to explore if Optima LMS is effective and efficient as it should be, by knowing how users can find information they need to complete tasks without help.

Of course I didn’t test every possible task that user could do with the system due to many constraints such as nature of research, limitation of human and monetary resources.

For this matter we created Scenarios and tasks to help us determine what we want to learn by observing six participants whom matched the testing profile to achieve tasks in the Work Informatics User Experience WIUX Laboratory in Turku ICT building.

Four kind of scenarios in total were planned for this testing, each one contained three tasks to be accomplished in a predetermined time frame,

each time participant was done with scenario he or she was given the post-task questionnaire and then the next scenario,

Tasks for testing Optima LMS are in this link https://optima.turkuamk.fi and it included:

Website layout: How easily can users understand the website layout?

Optima Login process Access to the desired degree program. Finding Information & help gained from the

Help icon. Navigation, Satisfaction & Dissatisfaction Advanced Options: number of object,

Cloud services…etc.