being human : future of human computer interaction
DESCRIPTION
Adopted from Microsoft Research "Being Human", this is a lecture on key points of future of HCI.TRANSCRIPT
Being HumanHuman computer interaction in 2020
Adopted from Microsoft Research on HCI, 2008
Some problems of today
Why Facebook does not automatically wish our friends during their birthday?
Why MMLS does not send us updates when a class is cancelled?
Why can’t we play DOTA or Cityville or throw Angry birds while driving?
How to solve this?
There have been various computer-driven revolutions in the
(see diagram below)
Photography, for example, has retained its familiarity despite
point of creation, people still ‘point and shoot’ in much the
1
3
2
4
activities – from buying food to paying our bills – and they
It is not just in terms of user experiences, such as shopping,
Four Computing Eras
1
3
2
4
1
3
2
4
1
3
2
4
1960s: Mainframe EraOne computer per many users.
1980s: Personal Computer EraOne computer per user.
2000s: Mobility EraSeveral computers per user.
2020 and beyond: Ubiquity EraThousands of computers per user.
14 15
There have been various computer-driven revolutions in the
(see diagram below)
Photography, for example, has retained its familiarity despite
point of creation, people still ‘point and shoot’ in much the
1
3
2
4
activities – from buying food to paying our bills – and they
It is not just in terms of user experiences, such as shopping,
Four Computing Eras
1
3
2
4
1
3
2
4
1
3
2
4
1960s: Mainframe EraOne computer per many users.
1980s: Personal Computer EraOne computer per user.
2000s: Mobility EraSeveral computers per user.
2020 and beyond: Ubiquity EraThousands of computers per user.
14 15
Solutions in 60s and 80sSolutions now and future
GUIs to Gestures
interface’ or GUI (for this and other terms used throughout, see
the Glossary)
developed that are richer and less prone to the many
there are tablet computers that use stylus-based interaction
on a screen, and even paper-based systems that digitally
shift to multiple points of input also supports novel forms
by gathering around it and interacting together (see the
‘Reactable‘, left)
input has also been developed that enables the movement
is beginning to be used to control digital movies, turn on
and our movements can be captured using CCTV cameras,
The Reactable: a multi-
touch interface for playing
music. Performers can
simultaneously interact
with it by moving and
rotating physical objects
on its surface. Reactable
was developed by Sergi
Jordà and colleagues at
the Universitat Pompeu
Fabra, Barcelona. Icelandic
songstress Björk used one
on her 2007 tour.
The HotHand device:
a ring worn by electric
guitar players that uses
motion sensors and a
wireless transmitter to
create different kinds of
sound effects by various
hand gestures.
16 17
Why can’t we play guitar with this?
GUIs to Gestures
interface’ or GUI (for this and other terms used throughout, see
the Glossary)
developed that are richer and less prone to the many
there are tablet computers that use stylus-based interaction
on a screen, and even paper-based systems that digitally
shift to multiple points of input also supports novel forms
by gathering around it and interacting together (see the
‘Reactable‘, left)
input has also been developed that enables the movement
is beginning to be used to control digital movies, turn on
and our movements can be captured using CCTV cameras,
The Reactable: a multi-
touch interface for playing
music. Performers can
simultaneously interact
with it by moving and
rotating physical objects
on its surface. Reactable
was developed by Sergi
Jordà and colleagues at
the Universitat Pompeu
Fabra, Barcelona. Icelandic
songstress Björk used one
on her 2007 tour.
The HotHand device:
a ring worn by electric
guitar players that uses
motion sensors and a
wireless transmitter to
create different kinds of
sound effects by various
hand gestures.
16 17
If GUI can become Gesture User interface from Graphical user interface, why not
guitar?
After all, things have changed
VDUs to Smart Fabrics
and plastic electronics are enabling displays to be made
materials to create very thin semi-conductive transistors that
can be embedded in all sorts of materials, from paper to cloth,
Electro-Mechanical Systems (MEMS), are also being made at an
sports, concerts and other cultural events are presented and
music players and other mobile devices providing information
dynamic readouts, reporting and alerting us to the status of
Handsets to the World in our Hands
mobile phones are rapidly becoming the most ubiquitous
provide a frustrating ‘genie-in-a-bottle’ experience – they
visualisation techniques – such as automatic zooming – have
Sensors of many types – from GPS location receivers and
accelerometers to RFID tag scanners – and some actuators
devices can also monitor our medical condition, too, such
device at a historic building and be offered up an audio or
the product came from, its associated air miles, and ecological
band’s poster and then sending it as a gift to a friend’s music
player can be as natural as a ‘cut and paste’ operation on a
Animated Textiles
developed by Studio
subTela at the Hexagram
Institute, Montreal, Canada.
These two jackets ‘synch
up’ when the wearers hold
hands, and the message
scrolls from the back of
one person to the other.
Talk to the hand…
phone ‘brick’ to the
latest Apple iPhone: as
the size reduces, the
potential expands.
18 19
VDUs to Smart Fabrics
and plastic electronics are enabling displays to be made
materials to create very thin semi-conductive transistors that
can be embedded in all sorts of materials, from paper to cloth,
Electro-Mechanical Systems (MEMS), are also being made at an
sports, concerts and other cultural events are presented and
music players and other mobile devices providing information
dynamic readouts, reporting and alerting us to the status of
Handsets to the World in our Hands
mobile phones are rapidly becoming the most ubiquitous
provide a frustrating ‘genie-in-a-bottle’ experience – they
visualisation techniques – such as automatic zooming – have
Sensors of many types – from GPS location receivers and
accelerometers to RFID tag scanners – and some actuators
devices can also monitor our medical condition, too, such
device at a historic building and be offered up an audio or
the product came from, its associated air miles, and ecological
band’s poster and then sending it as a gift to a friend’s music
player can be as natural as a ‘cut and paste’ operation on a
Animated Textiles
developed by Studio
subTela at the Hexagram
Institute, Montreal, Canada.
These two jackets ‘synch
up’ when the wearers hold
hands, and the message
scrolls from the back of
one person to the other.
Talk to the hand…
phone ‘brick’ to the
latest Apple iPhone: as
the size reduces, the
potential expands.
18 19
Brick to iPhone Smart fabric
Simple Robots to Autonomous Machines
That Learn
of assembly lines, as remote investigators of hazardous
have begun appearing in our homes as autonomous
vacuum cleaner has also become a commercial success;
of human companions, are also being commercialised,
partially due to their therapeutic qualities, being able to
is beyond the 2020 horizon, recent advances in machine-
robots to be programmed to respond and adapt to their
or not to notify a person of an incoming message, depending
on the nature and content (and therefore the urgency) of
commonplace and vexing volumes of spam that increasingly
While people are very much creatures of habit, they can also
be highly unpredictable and complex in their needs and
their intentions visible to each other: machines indicating
All of this proposes that humans and ‘intelligent’ machines
instructions given out by simple computers, this should
Emotional kitty: a robot
hardware platform called
iCAT uses a set of logical
rules to convey emotional
states as it makes decisions,
with the goal of improving
human-robot interaction.
It looks confused if it’s in
trouble, smiles if it gets
something right…
around us, costing next to nothing, from ten-a-penny memory
freely available for individuals to store their photos, videos,
Furthermore, huge amounts of information are being recorded
this may be erased after a period of time, some is stored more
The decreasing cost and increasing capacity of digital storage
production and sharing of digital content has also substantially
The Rovio robotic
webcam is wirelessly
connected to the Internet.
It roams around the home
providing an audio and
video link to keep an eye
on family or pets when
you’re out.
21
Dumb toy to emo robots
In its support of the doing of science, a challenge for the
development of computational tools and technologies is to
ensure that they are able to augment human reasoning and
We need to build tools that enable computing scientists and
other scientists to share and communicate their expertise
more integration of the computing and other sciences
are automated number-crunching tools that index, search
that model and highlight patterns, trends and anomalies in
based tools need to reveal and be explicit about their
computer simulates a complex system, does it simply create
millions of data points be represented in meaningful
computing resources and vast quantities of data, these issues
The ability to provide increasingly sophisticated tools to
and industriousness in our lives, and our aspirations for
data from the increasing array of sensors placed throughout
digital recordings that are being gathered and archived, such
Questions for interaction and design
tools be structured so they do not impede creative
and others to create tools for themselves to solve their
Questions of broader impact
Will computer-based tools eventually become so complex
•
•
•
•
•
•
and sensor-rich physical environments have been developed
extensive inquiries and decisions have been enabled, through
and far-reaching analyses, such as determining the effects of
constructive, be creative and, ultimately, be in control of our
content produced by all manner of people, from friends and
Augmenting human reasoning
Computers are increasingly being used to visualise and
In Vodafone’s vision of the
future, young musicians
will be able to create music
with friends in remote
places, all following or
creating a musical score
together. A wraparound
screen shows video images
of friends and displays the
digital score.
48 49
Won’t it be a fun desktop?
Some problems of today
Why Facebook does not automatically wish our friends during their birthday?
Why MMLS does not send us updates when a class is cancelled?
Why can’t we play DOTA or Cityville or throw Angry birds while driving?
Why can’t we just think and the assignment gets done?
Find ONE problem that YOU want to solve.
Write the problem, and what solution you want and why it matters?
I don’t expect you to know how to solve it.
15 minutes to think
human values, in all their diversity, be better understood and
been important to HCI: a focus on human-centred design,
and aspirations can be realised and supported through
From User Experience to Human Values
through their photo collections or post an image on a
might be about playing games; it might be about creating
Since its inception in the 1980s, HCI has been primarily
And, it tries to apply its methods to design good systems
different from the PCs, laptops or handheld computers of the
ourselves, be creative, and to nurture, protect, and care for one
computing or can HCI as an interdisciplinary community
of researchers, practitioners and designers become more
human values, including the moral and ethical aspects
reaching consequences for people beyond the immediate
enable anyone else in the street and beyond to permanently
behaviours, norms, and practices that are considered ethical
to design a responsible army robot or an ethical data-
propose a three-pronged approach that builds on its
user-centred research and design is conducted by including
another stage of conceptual analysis that explicitly addresses
these higher-level concerns, including the questions raised
been part of HCI, but that are equipped to address societal,
lingua franca
54 55
Being Human : Human Values
How do we solve the problems?
Stage 1: Understand
philosophy, psychology, art, literary theory, cultural
those involved in developing or designing the technology
photographs and other materials to preserve and honour
Build
Evalua
te
Study
Design
Extended user-centred,
model. The new stage
entails conceptual analysis
or ‘understanding’
The conventional
user-centred, four-stage
UnderstandStudy
Design
Build
Evalua
te
study, design, build and evaluate technology
page 59)
entails conceptual analysis
problem has traditionally been part of the study phase,
are thought through and the trade-offs examined in a
broader spectrum of designers (including architects and
Currently, the goal of a typical HCI research project is to
design or re-design a particular computing technology (be it
product, service, application, or system) in order to improve
upon or enhance a given experience (eg shopping online)
or to create a quite different experience than before (for
In both situations, initial research is conducted by learning
more about people’s current experiences (such as using
is judged to be enjoyable, pleasurable or valuable by the
The extended approach to HCI research and design is
intended to enable human values to be folded into the mix
can be entered into at any point, and usually iterates in the
59
Stage 1: Understand
philosophy, psychology, art, literary theory, cultural
those involved in developing or designing the technology
photographs and other materials to preserve and honour
Build
Evalua
teStudy
Design
Extended user-centred,
model. The new stage
entails conceptual analysis
or ‘understanding’
The conventional
user-centred, four-stage
UnderstandStudy
Design
Build
Evalua
testudy, design, build and evaluate technology
page 59)
entails conceptual analysis
problem has traditionally been part of the study phase,
are thought through and the trade-offs examined in a
broader spectrum of designers (including architects and
Currently, the goal of a typical HCI research project is to
design or re-design a particular computing technology (be it
product, service, application, or system) in order to improve
upon or enhance a given experience (eg shopping online)
or to create a quite different experience than before (for
In both situations, initial research is conducted by learning
more about people’s current experiences (such as using
is judged to be enjoyable, pleasurable or valuable by the
The extended approach to HCI research and design is
intended to enable human values to be folded into the mix
can be entered into at any point, and usually iterates in the
59
Previously New Suggestion
Understand
It is about what?
Play and creativity?
Social connectivity and togetherness?
Preserve and honour family history?
Reflect on personal past?
Supporting collaborative task in domestic situation?
Study
Stage 3: Design
The third stage is primarily a design or creative phase and
the design goal is to deepen our understanding of a set
technology is used or interpreted, or even contradict the
The design phase needs to consider the culture and place in
such different social and physical ecosystems as schools, stations,
design implications for existing infrastructures, the people that
not just the individual user experience of playing music, sending
Stage 2: Study
This stage of research consists of developing a deeper
or practices around a particular technology, the extended
those mechanics of interaction help people achieve long-
are manifest, too, in the hopes and ideals of people, the
involving input from disciplines outside of HCI, focusing on
This stage usually entails conducting a user study of one
on this, further user studies can be conducted to examine
shopper and an on-line process, this can be investigated in
Hence, Stage 2 provides a grounded understanding of
Design
Stage 3: Design
The third stage is primarily a design or creative phase and
the design goal is to deepen our understanding of a set
technology is used or interpreted, or even contradict the
The design phase needs to consider the culture and place in
such different social and physical ecosystems as schools, stations,
design implications for existing infrastructures, the people that
not just the individual user experience of playing music, sending
Stage 2: Study
This stage of research consists of developing a deeper
or practices around a particular technology, the extended
those mechanics of interaction help people achieve long-
are manifest, too, in the hopes and ideals of people, the
involving input from disciplines outside of HCI, focusing on
This stage usually entails conducting a user study of one
on this, further user studies can be conducted to examine
shopper and an on-line process, this can be investigated in
Hence, Stage 2 provides a grounded understanding of
Build Stage 5: Evaluate
laboratory evaluations to in situ
technique is appropriate comes, in part, from the design
values are related to, say, digital footprints, then evaluation
useful resources for, let us say, government monitoring or,
are designed to satisfy the desires of an individual may
the need to assess some of the longer term and more far-
reaching impacts of the design in question, and again this
values in the evaluation stage is necessary, though, since it
considerations may also suggest to HCI researchers that
other experts from other disciplines need to participate in
Stage 4: Build
It might, as another example, involve the creation of
everyday objects such as furniture, or parts of the built
example, micro-payment devices simply require proximity
Some interactions are distributed across different parts of
a physical-digital ecosystem consisting of various devices
To the contrary, if these systems pertain to human values
of some sort, then HCI must endeavour to explore and
The complexity of such hybrid systems might seem to
value is appropriate) then more polished engineering of
to be displayed on a nearby screen – or a novel set of
techniques for building too, such as virtual simulations
and even Wizard of Oz
In this approach, some function of the system can be
Whatever the technique or the technology used in this
Phidgets or ‘physical
widgets’ are building
blocks to help developers
construct physical user
interfaces. These arose
from a research project led
by Saul Greenberg at the
University of Calgary.
A new device being
evaluated in a long-
family kitchen.
implications for other values’
62 63
PrototypingProgrammingElectronicsPhysical Computing
Evaluate
Stage 5: Evaluate
laboratory evaluations to in situ
technique is appropriate comes, in part, from the design
values are related to, say, digital footprints, then evaluation
useful resources for, let us say, government monitoring or,
are designed to satisfy the desires of an individual may
the need to assess some of the longer term and more far-
reaching impacts of the design in question, and again this
values in the evaluation stage is necessary, though, since it
considerations may also suggest to HCI researchers that
other experts from other disciplines need to participate in
Stage 4: Build
It might, as another example, involve the creation of
everyday objects such as furniture, or parts of the built
example, micro-payment devices simply require proximity
Some interactions are distributed across different parts of
a physical-digital ecosystem consisting of various devices
To the contrary, if these systems pertain to human values
of some sort, then HCI must endeavour to explore and
The complexity of such hybrid systems might seem to
value is appropriate) then more polished engineering of
to be displayed on a nearby screen – or a novel set of
techniques for building too, such as virtual simulations
and even Wizard of Oz
In this approach, some function of the system can be
Whatever the technique or the technology used in this
Phidgets or ‘physical
widgets’ are building
blocks to help developers
construct physical user
interfaces. These arose
from a research project led
by Saul Greenberg at the
University of Calgary.
A new device being
evaluated in a long-
family kitchen.
implications for other values’
62 63Family Kitchen
Use UNDERSTANDING to solve
It is about what?
Play and creativity?
Social connectivity and togetherness?
Preserve and honour family history?
Reflect on personal past?
Supporting collaborative task in domestic situation?
Let’s solve some real HUMAN problem