1

1vrije Universiteit, 2VirTouch Ltd, 3The Mediator Group, 4Exact Holding NV

Johan F. Hoorn1, Simon van Dam2, Guido Fambach3, Arco van Nieuwland4, Gerrit C. van der

Veer1

Looking for Win/Win Solutions between Lab-work and Hands-on

Experience in IT Research

2004 /15

2

Contents Status Goal Problem Analysis Solution Three cases Conclusions Recommendations Questions

Mens-MachineInteractie

/15

3

Status

Postdoc-project 2001-2005 Supervisors: Hans van Vliet and Gerrit van der Veer Three international publications, one submitted Three empirical studies in progress, two pending

/15

4

Goal

To arrive at IT research that is

Scientifically reliable Valid Relevant to business goals

/15

5

Problem

Science Business/15

6

Analysis

/15

7

Solution

Field experiments combine

Structured observations / lab methods Random assignment to conditions Real situations (people and systems) At the cost of studying single cases

/15

8

Three cases

Simon van Dam and VTPlayer user Stevie Wonder

Contrasts are made touchable

Text is translated into Braille

Two 4*4 pin pads

/15

9

Which blind pupils of Bartimeus Zeist are the faster and/or more effective learners; those freely exploring geographic materialwith the VTplayer or those with a VTPlayer and a teacher?

Problem: A pretest on what someone already knows has effects onknowledge activation, and thus, on the results of working with theapplication

Solution: Solomon’s four groups design

Dependents: Speed and effectiveness of learning geography

User group 1: Pretest - Teacher+ VTPlayer - PosttestUser group 2: Pretest - VTPlayer - PosttestUser group 3: - Teacher+ VTPlayer - PosttestUser group 4: - VTPlayer - Posttest

Effect ofpreviousknowledge

Effect ofteachingmethod

interaction

/15

10

Question for your exams:

In view of Solomon, whatis wrong with the student’sknowledge assessment withinthe Didactor method?

User group 1: Pretest - Teacher+ Didactor - Posttest

User group 2: Pretest - Didactor - PosttestUser group 3: - Teacher+ Didactor - PosttestUser group 4: - Didactor - Posttest

Three controlconditions aremissing /15

11

Efficiency:

Trade-offbetweenspeed andaccuracy

measured in

clock-timevs. error rate

in relation to

perceivedspeed vs.perceivedaccuracy

System

User

Measuring the efficiency(and possibly effectiveness)of certain business processes before andafter introduction of the e-Synergy platform /15

12

ConclusionsImproved cooperation between business and science enhances performance Money for quality information

Both need ecologically valid and relevant data (business approach) Study real business cases

Both need reliable and accurate data (scientific approach) Experimental designs and measurements

/15

13

Recommendations

Business

Provide a research field Provide supplies (assistants, software) Don’t hush up business problems Don’t try to bias results The time-line is NOT sacred

/15

14

Recommendations

Science

Don’t expect lab-conditions Be a better experimenter/analyst instead Conduct hit-and-run research Don’t interfere with the business The business’s time-line IS sacred

/15

15

Questions?

/15