design of a pilot experiment for the evaluation of automated persuasion systems marco guerini...

- WP8 workshop, Trento, 17/18 Nov.- WP8 workshop, Trento, 17/18 Nov.

Design of a Pilot Experiment for the Evaluation of Automated Persuasion systems

Marco Guerini (ITC-Irst)

Shlomo Hareli (University of Haifa)

Shy Tzafrir (University of Haifa)

2


General Motivations

Persuasive interfaces need to be evaluated: they have to be effective with real users.

Evaluation studies up to now have focused mainly on usability.

Evaluation of effectiveness is not straightforward, specific methodologies have to be used / defined.

3


Approach

Use of experiments/measures coming mainly from Social Psy.

Specific Question: Can display of emotion enhance the persuasive force of a message in HCI?

4


Specific Motivations

2 specific interests:

Automated persuasive communication: evaluation study of the Promoter Prototype, developed at Irst.

Emotions in persuasive comm.: study of the Fitting Principle (Univ. of Haifa) in persuasive Computer-Human Communication

5


Promoter prototype

Content Ordering Meta-Strategies (COMS). Given basic persuasive strategies, how to compose them to create a complex message.

Example of basic persuasive strategy on positive consequences: “action a has the positive outcome o”.

COMS1 a message made with positive pieces of information. As a pilot experiment we will start from this one.

COMS2 based on framing effect theory (positive consequences of the required action following negative consequences of a conflicting action)

etc…

6


The Fitting Principle

The theory we use for emotion displaying is the Fitting principle (University of Haifa):

A message that is accompanied by an emotional expression that fits the valance of the message will be more persuasive than the same message emitted with no emotion or an emotional expression that contradicts the message.

7


Advantages

an evaluation study of the specific persuasion system (Promoter)

a contribution to the design of a general model for evaluation of automated persuasion systems

8


Conditions of the Experiment

The experiment will study one persuasive meta-strategy (i.e. COMS1) on 3 different conditions (between subjects, each person gets only one condition). 3 conditions are:

Fitting emotion Unfitting emotion Neutral, no emotion, just message

9


The message

You should come and visit the renewed exhibition on the archaeology of the Land of Israel at the Hecht Museum. It has pieces from the Chalcolithic, through the Hellenistic, Canaanite up to the Byzantine Period. This is a good opportunity to enlarge your knowledge about Israel History, to learn nice anecdotes to tell your friends and it is really easy to reach, since it is inside your own university!

10


Fitting Clip

11


Unfitting Clip

12


Neutral Clip

13


Expected Results

The Fitting message is more persuasive than the other messages. The unfitting message is less persuasive relative to the other messages.

14


Conclusions (1)

Evaluation of Persuasive interfaces: specific methodologies have to be used / defined.

Design of an experiment using measures coming from Social Psy.

15


Conclusions (2)

Specific question: Can display of emotion enhance the persuasive force of a message in HCC?

Advantages: evaluation study of the specific system (Promoter) & contribution for a general model for evaluation of automated persuasion systems

16


Future work (1)

Preamble: We can expect, from studies in HHI and HCI, that each persuasive strategy alone works (COMS, fitting princ.)

Broader result: not just the replication of studies already done. Test the interaction of different strategies:

there is no results about their mixing.

17


Future Work (2)

Promoter uses four main content-ordering persuasive strategies: the experiment replicated conceptually three times, one for each strategy (after COMS1, also COMS2, COMS3, COMS4).

Study the interaction between fitting principle and COMS strategies.

18


Ethical Issues

Challenge 1: agents that are ethically aware (i.e. capable of reasoning on the ethicality of their actions and possibly to correct their behaviour)

Challenge 2: a “layered” model of ethical persuasion: ethicality of the act of persuading + ethicality of the induced action

Ethical capabilities to be encoded at the meta-reasoning level

design of a pilot experiment for the evaluation of automated persuasion systems marco guerini...

Documents