Author: B.Sc. Aapo LumikoivuSupervisor: D.Sc. Kalevi KilkkiInstructor: M.Sc. Åsa Enberg

Aalto-UniversitySchool of Electrical EngineeringDepartment of Communications and NetworkingDegree program of Human Factors in Communications Design

4.9.2011

2 / 18

• Introduction to the thesis• Key concepts• Research questions• Methodology• Development process• Results• Successes• Shortcomings• Summary• Sources• For the interested

3 / 18

• The thesis is a report of a product development project, where an instruction manual for a research tool was created.

• The development happened during May – June 2011, for the Transportation Engineering Research Group of Aalto University.

• The development project was user-centered and utilized 16 real end-users in three test phases.

• Each test phase was followed by a design iteration, where the content and structure of the instructions were altered.

4 / 18

• The development was guided by experience based usability heuristics and theory based cognitive engineering principles.

• The purpose of the development was to allow access to the measurement system for people with or without prior knowledge or expertise about the system.

• The end result is an intuitive instruction manual, which promotes fast learning and proper operation of a complex system.

5 / 18

• An instruction manual facilitates proper and safe operation of a product, by offering information to the user.

• An instrumented research vehicle is a moving measurement platform for transportation and highway engineering.

• Usability is the effectiveness, efficiency and satisfaction related to using a product in a specific context. [1]

• User-centered design (UCD) is a research and development (R&D) concept, which either…• … uses specialist methods, where the designer systematically analyses the product from the

user’s perspectiveor• … uses real end-users in a participatory development process.

• User-experience is the personal perception and experience of use, which forms prior, during and after using a product. [1]

6 / 18

UCD: User-centered design R&D: Research and Development

• What is the definition of usability in the case of an instruction manual?• How to incorporate users into the development of a manual?• How to ensure the usability of the manual?• What are the benefits of the UCD-approach in this case??• What is a suitable direction for further development of the manual?

7 / 18

• Design guidelines from experience based heuristics and theory based cognitive engineering principles:• Nielsen’s Usability heuristics [2]• Lund’s usability maxims [3]• Heuristics of choice [4]• Gerhard-Powal’s cognitive principles [5].

• Participatory iterative user-centered design:• Real end-users used throughout the project• A round of testing followed by a design iteration• Initial design from a task analysis specialist method• Two rounds of scenario testing with 5 + 5 participants• A single Focus group discussion with 10 participants

8 / 18

9 / 18

• Selecting design guidelines and a structure for the development were the first steps in the process

• A task analysis produced material for the initial version of the manual• A scenario-of-use test was conducted with 5 people and the results were used

to improve the manual and create material for further testing• A focus group discussion between a moderator and 10 people was used to

acquire a more wider view of the issues related to manual usability• The third version of the manual was once again subjected to scenario testing

with 5 new users• The release version was finalized after 4 rounds of testing and iteration• The direction for future development was drafted based on the whole process

and user feedback.

Date Milestone Description

2.5 Beginning Project timeline and testing structure finalized

9.5 Task analysis Hierarchical task analysis begins

12.5 Version 1 Initial content and structure completed for the manual

13.5 Pilot test A pilot test for scenario-of-use testing

17-19.5 Scenario 1 First round of scenario tests with manual version 1

30.5 Version 2 First iteration of the manual completed

9.6 Focus group Focus group discussion with manual version 2

13.6 Version 3 Second iteration of the manual completed

14-16.6 Scenario 2 Second round of scenario tests with manual version 3

8.7 Release version Final iteration of the manual completed

10 / 18

11 / 18

• The instruction manual was finished on time and has just been taken into use (August 2011)•A definition for the usability of an instruction manual:• Effectiveness: the information content and structure of the manual must facilitate positive and

expected outcomes of the usage• Efficiency: the required investment in time and effort to search information from the manual

must be negligible compared to the time required to perform the actual task• Satisfaction: the usage gives the user a feeling of satisfaction by generating feelings of aptitude

and “being in control”.

•The participatory iterative testing method was cost-effective and well suited for incorporating users into a R&D project•The usability of the product was ensured by methodically abandoning poor solutions and gradually introducing corroborated improvements•The UCD approach ensured that the end-result is significantly more usable than the original design•The test participants suggested a score of further improvements.

12 / 18

UCD: User-centered design R&D: Research and Development

• The release version is substantially more usable than the initial design• Incorporating users into the project helped mitigate the lack of a designer team• The initial iterative development process minimized the amount of work that needed to be abandoned• The users were able to familiarize themselves with the vehicle and the manual in a meaningful context• The flaws and deficiencies in the manual should be revealed by the feedback form incorporated into the manual.

13 / 18

• The usability of the manual may still be poor in some areas• The design process produced a task oriented manual, other approaches were not even considered• The final version was not subjected to user testing…

… but the iterative process needed to stop somewhere• There are other shortcomings as well, i.e. the kind that can only be found by using the manual. Hopefully the users will use the feedback forms to convey encountered problems.

14 / 18

• A participatory user-centered design process was used to create an instruction manual• The iterative process ensured that poor solutions were abandoned throughout the project with minimal loss of work• The use of real end-users was cost-effective and produced good results• The release version of the manual is in use and material to further improve the manual is being collected.

15 / 18

[1] International Standard ISO 9241-210. Ergonomics of human-system interaction - Human-centered design for interactive systems. Geneva : International Organization for Standardization, 2010.[2] Nielsen, J. Usability engineering. San Diego : Academic press, 1993.[3] Lund, A M. Expert ratings of usability maxims. Ergonomics in design. 1997, Vol. 5, 3, pp. 15-20.[4] Tversky, A and Kahneman, D. Judgment under Uncertainty: Heuristics and Biases. Science. 1974, Vol. 185, 4157, pp. 1124-1131.[5] Gerhardt-Powals, J. Cognitive engineering principles for human-computer performance. International Journal of Human-Computer Interaction. 1996, Vol. 8, 2, pp. 189-211.

16 / 18

• The Transportation Engineering Research Group of Aalto University: http://civil.aalto.fi/en/research/transportation/• Jakob Nielsen’s book on Usability Engineering: http://www.useit.com/jakob/useengbook.html• An endless source of information about Human-Computer Interaction: http://www.interaction-design.org/

17 / 18

18 / 18