applying iterative design to the eco-feedback design process

Applying Iterative Design to the Eco-Feedback Design Process

Human Computer Interaction Laboratory

@jonfroehlich Assistant Professor Computer Science

Behavior, Energy, & Climate Change Conference Nov 13, 2012

eco-feedback sensing and visualizing behavior to reduce environmental impact

eco-feedback sensing and visualizing behavior to reduce environmental impact

“Getting the design right and the right design.”

— Bill Buxton

Sketching User Experiences

Before moving forward, I want to ask a question…

The following eco-feedback paper is missing something.

What is it?

Brandon et al., Reducing Household Energy Consumption: A Qualitative & Quantitative Field Study, Environmental Psychology 1999

Brandon et al., Reducing Household Energy Consumption: A Qualitative & Quantitative Field Study, Environmental Psychology 1999

½ paragraph description of

eco-feedback interface and no

screenshots in 11 page paper

Brandon et al., Reducing Household Energy Consumption: A Qualitative & Quantitative Field Study, Environmental Psychology 1999

Well, but that was 1999.

Grønhøj & Thøgersen, Feedback on Household Electricity Consumption: Learning & Social Influence Processes, IJCS2011

Grønhøj & Thøgersen, Feedback on Household Electricity Consumption: Learning & Social Influence Processes, IJCS2011

2 paragraph description of

eco-feedback interface and no

screenshots in 8 page paper

Where’s Waldo?

So, clearly a disciplinary divide…

psychologists

designers

engineers

economists

building scientists

others?

This oversight seems to reflect a lack of recognition

about the critical role that particular design

choices play in affecting behavior.

Jain et al., Assessing Eco-Feedback Interface Usage and Design to Drive Energy Efficiency in Buildings, Energy and Buildings 2012

Jain et al., Assessing Eco-Feedback Interface Usage and Design to Drive Energy Efficiency in Buildings, Energy and Buildings 2012

Perhaps because of the design de-emphasis, very few

papers discuss the design process that led to the

ultimate design artifact that was created and studied

This is abigproblem

How can we structure and support the design process to create and identify the most promising (and potentially most influential) aspects of an eco-feedback design robustly and in a cost-efficient manner?

Overarching Question

Ideation Data

Gathering

Ideation /

Sketch

Pilot

Studies Refinement

Formative

Evaluation Refinement

Small Field

Deployment(s) Refinement

Large

Randomized

Control Trial(s)

An Eco-Feedback Iterative Design Process

A/B Design

Ideas Pilot

Studies Refinement

A/B Testing

RCTs

Gunel, A. The Halo Effect: Using Behavior to Upgrade Technology,BECC2012

Ideation Data

Gathering

Ideation /

Sketch

Pilot

Studies Refinement

Formative

Evaluation Refinement

Small Field

Deployment(s) Refinement

Large

Randomized

Control Trial(s)

A/B Design

Ideas Pilot

Studies Refinement

A/B Testing

RCTs

An Eco-Feedback Iterative Design Process A/B Testing is Ultimate Playground

Ideation Data

Gathering

Ideation /

Sketch

Pilot

Studies Refinement

Formative

Evaluation Refinement

Small Field

Deployment(s) Refinement

Large

Randomized

Control Trial(s)

A/B Design

Ideas Pilot

Studies Refinement

A/B Testing

RCTs

A/B Testing is Ultimate Playground

Gunel, A. The Halo Effect: Using Behavior to Upgrade Technology,BECC2012

Ideation Data

Gathering

Ideation /

Sketch

Pilot

Studies Refinement

Formative

Evaluation Refinement Small Field

Deployment(s) Refinement

Large

Randomized

Control Trial(s)

A/B Design

Ideas Pilot

Studies Refinement

A/B Testing

RCTs

Evaluating early design ideas to prepare for

field deployments

An Eco-Feedback Iterative Design Process

Jon Froehlich

Leah Findlater

Marilyn Ostergren

Solai Ramanathan

Josh Peterson

Inness Wragg

Eric Larson

Fabia Fu

Mazhengmin Bai

Shwetak N. Patel

James A. Landay

Froehlich et al., The Design & Evaluation of Prototype Eco-Feedback Displays for Fixture-Level Water Usage Data, CHI2012

Ideation Data

Gathering

Ideation /

Sketch

Pilot

Studies Refinement

Formative

Evaluation Refinement

Small Field

Deployment(s) Refinement

Large

Randomized

Control Trial(s)

Goal: gather formative data and use as basis to create a set of early, promising designs

Inquiry Methods: ethnography, interviews, surveys, literature reviews

An Eco-Feedback Iterative Design Process

Ideation Data

Gathering

Ideation /

Sketch

Pilot

Studies Refinement

Formative

Evaluation Refinement

Small Field

Deployment(s) Refinement

Large

Randomized

Control Trial(s)

Goal: gather formative data and use as basis to create a set of early, promising designs

Inquiry Methods: ethnography, interviews, surveys, literature reviews

An Eco-Feedback Iterative Design Process

Informal interviews with water experts (e.g., SPU, Amy Vickers)

Literature review of water resource management, environmental psychology

Our own online survey of water usage attitudes & knowledge (N=656 respondents)

Respondents (N=651) dramatically underestimated the

amount of water used in common everyday activities.

toilet :

shower :

bath :

low-flow shower :

outdoor yard watering :

by 15%

by 30%

by 55%

by 60%

by 83% to 95%

underestimate

[Froehlich, UW PhD Dissertation, 2011]

Poor Water Literacy

Challenge: how can we use gathered data to inform our designs?

webpage in-home display

mobile phone app

wearable interface

custom display

manifestation

size small large

ambience not-ambient ambient

DISPLAY MEDIUM

update frequency real-time monthly or less

effort to access low high

spatial proximity to behavior

attentional demand glanceable high attention

co-located remote

INFORMATION ACCESS

user poll

interface customizability none high

degree of interactivity none high

INTERACTIVITY

user annotations user corrections user

additions

aesthetic pragmatic artistic

visual complexity simple complex

primary visual encoding textual graphical

data granularity coarse-grain fine-grain

time window <hour >year

DATA REPRESENTATION

primary view temporal spatial categorical

measurement unit resource metaphor cost environmental

impact time

temporal grouping ≤sec ≥year by hour by day by week by month

data grouping by

resource by consumption

category by

person by

time by

space

activity

by activity

target person

social- comparison

data sharing none everyone

private/ public

SOCIAL ASPECTS

household community state country

private public

available unavailable (see COMPARISON) degree of

actionability low high

decision support

personal-ization no personalization highly personalized

ACTIONABILITY/UTILITY

suggests actions

anomaly alerts

suggests purchase decisions

automation/ control no control system controls resource use

information intent Informs one action informs many actions

MOTIVATIONAL/PERSUASIVE STRATEGIES

persuasive tactics from psychology and applied social

psychology disciplines:

persuasive design persuasive technology

behavioral science/economics environmental psychology

game design social marketing

health behavior change

persuasive tactics include:

rewards punishment public commitment written commitment loss aversion kairos encouragement descriptive norms scarcity principle framing anchoring bias defaults

goal-setting narrative likeability reputation competition social proof authority emotional appeals door-in-face unlock features endowment effect collection building

comparison by time past projected

COMPARISON

comparison target self goal social

difficulty to reach comparison target easy hard

time window

time granularity

data grouping

data granularity

measurement unit

statistic

raw value average median mode other

comparison variables computation

@ this time [yest, last wk, mo, yr] [hrly, daily, wkly, monthly, yrly] over past [X] days this day type [weekday, weekend] this day of week (e.g., mondays)

social-comp. target

geographically proximal

selected social network

demographically similar

goal-setting strategy self-set externally-set system-set

Eco-Feedback Design Space

[Froehlich et al., HCIC2009; CHI2010; UW PhD Dissertation 2011]

My own experiences Existing frameworks (e.g., persuasive tech, health informatics,

other eco-feedback work, and infovis)

Psychology (e.g., environmental psychology, social

psychology, behavioral economics)

webpage in-home display

mobile phone app

wearable interface

custom display

manifestation

size small large

ambience not-ambient ambient

DISPLAY MEDIUM

update frequency real-time monthly or less

effort to access low high

spatial proximity to behavior

attentional demand glanceable high attention

co-located remote

INFORMATION ACCESS

user poll

interface customizability none high

degree of interactivity none high

INTERACTIVITY

user annotations user corrections user

additions

aesthetic pragmatic artistic

visual complexity simple complex

primary visual encoding textual graphical

data granularity coarse-grain fine-grain

time window <hour >year

DATA REPRESENTATION

primary view temporal spatial categorical

measurement unit resource metaphor cost environmental

impact time

temporal grouping ≤sec ≥year by hour by day by week by month

data grouping by

resource by consumption

category by

person by

time by

space

activity

by activity

target person

social- comparison

data sharing none everyone

private/ public

SOCIAL ASPECTS

household community state country

private public

available unavailable (see COMPARISON) degree of

actionability low high

decision support

personal-ization no personalization highly personalized

ACTIONABILITY/UTILITY

suggests actions

anomaly alerts

suggests purchase decisions

automation/ control no control system controls resource use

information intent Informs one action informs many actions

MOTIVATIONAL/PERSUASIVE STRATEGIES

persuasive tactics from psychology and applied social

psychology disciplines:

persuasive design persuasive technology

behavioral science/economics environmental psychology

game design social marketing

health behavior change

persuasive tactics include:

rewards punishment public commitment written commitment loss aversion kairos encouragement descriptive norms scarcity principle framing anchoring bias defaults

goal-setting narrative likeability reputation competition social proof authority emotional appeals door-in-face unlock features endowment effect collection building

comparison by time past projected

COMPARISON

comparison target self goal social

difficulty to reach comparison target easy hard

time window

time granularity

data grouping

data granularity

measurement unit

statistic

raw value average median mode other

comparison variables computation

@ this time [yest, last wk, mo, yr] [hrly, daily, wkly, monthly, yrly] over past [X] days this day type [weekday, weekend] this day of week (e.g., mondays)

social-comp. target

geographically proximal

selected social network

demographically similar

goal-setting strategy self-set externally-set system-set

Eco-Feedback Design Space

[Froehlich et al., HCIC2009; CHI2010; UW PhD Dissertation 2011]

behavioral

models

social

inputs

data

representation information

access

display

medium

comparison

actionability

motivational

strategies

eco-feedback design space

the

behavioral

models

social

inputs

data

representation information

access

display

medium

comparison

actionability

motivational

strategies

eco-feedback design space

the

social

inputs

data

representation information

access

display

medium

comparison

actionability

motivational

strategies

eco-feedback design space

the

behavioral

models

social

data

representation information

access

display

medium

comparison

actionability

motivational

strategies

eco-feedback design space

the

behavioral

models

inputs

behavioral

models

social

inputs

data

representation information

access

display

medium

comparison

actionability

motivational

strategies

eco-feedback design space

the

social

inputs

data

representation information

access

display

medium

comparison

actionability

motivational

strategies

eco-feedback design space

the

behavioral

models

inputs

social

behavioral

models

data

representation

information

access

display

medium

comparison

actionability

motivational

strategies

eco-feedback design space

the

pragmatic artistic aesthetic

simple complex visual complexity

coarse-grain fine-grain data granularity

primary view temporal spatial categorical

measurement unit resource cost metaphor environmental

impact time activity

< hour > year time granularity

inputs

social

behavioral

models

data

representation

information

access

display

medium

comparison

actionability

motivational

strategies

eco-feedback design space

the

pragmatic artistic aesthetic

simple complex visual complexity

coarse-grain fine-grain data granularity

primary view temporal spatial categorical

measurement unit resource cost metaphor environmental

impact time activity

< hour > year time granularity

coarse-grain fine-grain ≥neighbor- hood

home room fixture category

fixture ≤ valve activity

Data Granularity

Higher

fidelity

mockups

Sketches

Iterative Design Fidelities Prototype and Evaluate Across

a Fidelity Spectrum

Sketch Lo-to-Mid Fidelity

Mockup

Higher Fidelity

Mockup

Ideation Data

Gathering

Ideation /

Sketch

Pilot

Studies Refinement

Formative

Evaluation Refinement

Small Field

Deployment(s) Refinement

Large

Randomized

Control Trial(s)

Goal: gather formative data and use as basis to create a set of early, promising designs

Inquiry Methods: ethnography, interviews, surveys, literature reviews

An Eco-Feedback Iterative Design Process

Ideation Data

Gathering

Ideation /

Sketch

Pilot

Studies Refinement

Formative

Evaluation Refinement

Small Field

Deployment(s) Refinement

Large

Randomized

Control Trial(s)

Goal: test early design ideas, improve promising ideas, improve usability / aesthetic

Evaluation Methods: online interactive surveys, design probe-based interviews, lab studies

Challenge: the ultimate goal is to create a design that informs and, possibly, motivates

behavior. The former is easier to evaluate with these methods than the latter.

An Eco-Feedback Iterative Design Process

Online Survey Recruitment o Online postings and word-of-mouth

Survey Design o 63 questions (10 optional)

o Question and answer order

randomized when possible

Collected Data o 712 completed surveys

(651 from US or Canada)

o Nearly 6,000 qualitative responses

Our online interactive survey allowed us to

study a large N and gather both quantitative

and qualitative data

Comparisons were the most

uniformly desired pieces of

information of all the dimensions

Self-comparison

was most preferred

91%

Our in-home, design-probe interviews allowed us

to explore how the display was received by families

and how (and where) it fit in a domestic setting

In-Home Interviews Recruitment o Online postings and word-of-mouth

o Specifically recruited families

Interview Method o Semi-structured with two researchers

o 90-minutes, 3-phases

o Data coded by two researchers into themes

Participants o 10 households (20 adults)

o 11 female/9 male

o Diff. socio-economic backgrounds & occupations

o 18 had college degrees

Display Location Preferences

kitchen

near thermostat

high traffic areas

accessible when needed

Behavioral Lab Study

Ideation Data

Gathering

Ideation /

Sketch

Pilot

Studies Refinement

Formative

Evaluation Refinement

Small Field

Deployment(s) Refinement

Large

Randomized

Control Trial(s)

Integrate Findings & Revise Designs

A/B Design

Ideas Pilot

Studies Refinement

A/B Testing

RCTs

Place more emphasis on describing eco-feedback

designs and how design choices may affect behavior

in our research papers / white papers

Help generate reusable design knowledge by

including information not just on the final eco-

feedback design but the process used to achieve it

1

2

A Call

Froehlich et al., The Design and Evaluation of Prototype Eco-Feedback Displays for Fixture-Level Water Usage

Data, CHI2012, http://bit.ly/JonUMDPubs

Froehlich et al., The Design & Evaluation of Prototype Eco-Feedback Displays for Fixture-Level Water Usage Data, CHI2012

More detail and examples about the eco-feedback

design process here: http://bit.ly/jonUMD

Jon Froehlich, PhD Dissertation, 2011, http://www.cs.umd.edu/~jonf/publications.html

Jon Froehlich, Sensing and Feedback of Everyday Activities to Promote Environmental Behaviors, PhD Dissertation 2011

Jon Froehlich, Moving Beyond Line Graphs, BECC2010, http://www.cs.umd.edu/~jonf/talks.html

Applying Iterative Design to the Eco-Feedback Design Process

Human Computer Interaction Laboratory

@jonfroehlich Assistant Professor Computer Science

http://bit.ly/jonUMD jonf@umd.edu