Introduction to

MS3305 New Media Theory/Practice: User Experience

Design

MODULE LEADER: TONY SAMPSONRoom EB1.31

Email: t.d.sampson@uel.ac.ukOffice Hours: By email appointment

Module InfoWhere are we?

• MS 3305 – NEW MEDIA THEORY-PRACTICE: USER EXPERIENCE DESIGN

• Tuesday Lecture 11.00 – 12.00EB.3.17• Seminar in 12.00 – 2.00 WB.2.05• User testing can be booked in the emotionUX lab in the MPS

• http://ms3305.blogspot.com/– Lecture notes

• http://homepages.uel.ac.uk/T.D.Sampson/DocLand/MS3305/MS3305.htm– Full module guide

Orientation

The first lecture will map out the module with the subsequent seminar providing an opportunity to discuss what you can expect to get out of it, and what you need to put in.

First lecture introduces a module debate

Aims of the module• Main aim(s) of the module:

• To explore ideas, debates and theories in the field of new media relating to media consumption, attention, emotion and cognition, memory, the “user”, usability, user experience design, HCI and neuromarketing.

• To integrate theoretical ideas and research focused practice

• To investigate the relationship between theory and practice in the new media field in general and in the context of user experience specifically

Main topics of study• Concepts of new media theory and their relation

to research and practice

• Theoretical approaches to new media, including media consumption, labour, attention, emotion and cognition, memory, the “user”, usability, user experience design, HCI and neuromarketing.

• Practical approaches to prototyping, testing and other user centred research methods.

• The social and cultural power relations implicated in new media consumer/producer relations

Assessment

• You are required to submit two pieces of course work. These course works are separate but related. Together they account for all available marks.

• CW1 Illustrated Essay of 1,500 words - with a minimum of 5 good quality printed colour images (50%)– Choose one title from

five titles• CW2 • Media/Multimedia

Prototype (50%)– To accompany a major

project

Illustrated Essay (50%) choose 1 from the 4 listed below

1. Apply Crary’s concept of the attentive subject to current marketing attempts to draw attention to products and brands? In your discussion refer to the techniques and theories used in neuromarketing.

2. Referring to Harrison et al’s Three Paradigm of HCI, and other relevant sources, discuss the continuities and discontinuities between all three paradigms. In your discussion consider what we have called the politics of HCI, particularly the relation HCI has with, for example, capitalism and work.

3. Use Norman’s model of experience processing to explain how consumers become emotionally and affectively connected to the brands and products they consume? Refer to an example, like Apple, to support and illustrate your discussion.

4. Affect and emotions are increasingly understood to influence cognitive processes such as decision making and memory. Discuss how the relations between affect, feeling, emotion and cognition is grasped and how it relates to consumption. Refer to a good mixture of theories, for example, from psychology, neuroscience, design, and cultural theory.

• All written essays to use images to illustrate and support your analysis. This must be included in the layout of the printed essay.

CW2 Media/Multimedia Prototype (50%)

• You are asked to produce a “paper” prototype that can be used in the design of a larger project (MS2306/MS3308?).

• The prototype must: 1. Adhere to methods introduced in the seminar sessions (or justify other

methods)2. Be logical and conform to stages of UXD design (user analysis, design

concepts and implementation)3. Include evidence of user testing4. Demonstrate iterative modifications based on documented user tests5. Engage in a creative way with the theoretical ideas discussed in the

module (i.e. feature elements of criticality built into the design).

The submission will take the form of (a) a slide presentation providing supporting evidence of the above, and (b) an actual paper prototype.

Deadlines• CW1: Illustrated Essay

of 1,500 words (50%)

• Final Deadline: Submit ONE hardcopy of CW1 to the Student Enquiries Desk

• Deadline: before 4pm Tues April 22nd

• CW2: Media/Multimedia Prototype (50%)

• Draft Deadline 1: • Email plan for CW2

presentation to the module leader

• Deadline 1: End of day April 22nd

• Final Deadline 2: Submit ONE copy of CW2 presentation to the Student Enquiries Desk!

• Deadline 2: before 4pm Tuesday May 13th

Week-by-Week• Lectures

• The theoretical frameworks– Debate– Prototyping/Three

Paradigms – HCI focus on attention– Attentive subjects (2

weeks)– Recap on Affect – Persuasion– User experience

design/economy

After student vacation focus on coursework

• Seminars

• The practical work

– Stages– Methods– Demonstrations– Presentations

Week-by-Week

• Week One: Today 4th Feb 2013

• Introducing Assessments and Context

• Lecture and Seminar: The Module Debate

• The Politics of User Testing• Reading Tony’s CTheory Paper

Week-by-Week

• Lecture Series and Seminars begins

• Week Two 11th Feb

• Lecture: The Politics of HCI: Human Machine Coupling and the Prototype (continuing from MS2306)

• Seminar: The Stages of User Testing

Week-by-Week

• Week Three 18th Feb

• Lecture - The Attentive User • HCI and Approaches to User Testing – Focus on

Cognitive Framework - HCI by Jenny Preece et al

• Seminar: Scoping the Prototype Project • Learning from Users

Week-by-Week

• Week Four 25th Feb

• Lecture - Crary’s Cultural History of Attention: The Attentive Subject

• Seminar: Scoping the Prototype Project • Looking at Users

Week-by-Week

• Week Five March 4th

• Lecture - Crary’s Cultural History of Attention: Pathologies of Inattention, Freewill and Media Hypnosis

• Seminar: Scoping the Prototype Project• Asking Users

Week-by-Week• Week Six March 11th

• Lecture - Recapping Affect and Looking at Persuasion (advancing from MS2306)

• Seminar: Scoping the Prototype Project• Users Trying Out Experiences

Week-by-Week

• Week Seven March 18th

• Lecture - The User Experience Economy

• Seminar: Focus on approach to prototype – what methods have you used and why? (Tony and Mary)

• Group discussion• What to include in the essay?• What to include in the presentation plan?

Week-by-Week

• Focus shifts to Assessments and Presentations

• Week Eight March 25th

• Entire session dedicated to essay development (pre booked tutorials with Tony)

• Groups to visit emotionUX lab to use kit (Mary)

Week-by-Week

• Week Nine April 1st

• Entire session dedicated to essay development (pre booked tutorials with Tony)

• Groups to visit emotionUX lab to use kit (Mary)

Week-by-Week

• Vacation (week starting April 7th and week starting April 14th)

Week-by-Week

• Week Ten April 22nd

• CW1 Illustrated Essay Deadline • Office drop in session (Mary and Tony)• CW2 Draft Prototype Deadline (plan for

presentation to be emailed to Tony before end of day)

Week-by-Week

• Week Eleven April 29th

• Demonstration of Prototype Assessments

Week-by-Week

• Week Twelve May 6th

• Demonstration of Prototype Assessments

• Week Thirteen May 13th

• Review And Evaluation • CW2 Final Prototype hand in deadline • New Media Student Show (15th and 16th May)

Key readings for the essay• Harrison, S. Tatar D, and Sengers, P (2007) “The Three Paradigms of HCI,” Proceedings of CHI, San Jose, CA,. Archived at:

http://people.cs.vt.edu/~srh/Downloads/TheThreeParadigmsofHCI.pdf (accessed 5th April, 2011).• Chapter one of Crary, J Suspensions of Perception: attention, spectacle, and modern culture. The MIT Press, Cambridge, MA, 1999• Picard R W (1999) “Affective Computing for HCI”. Proceedings of HCI International (the 8th International Conference on Human-Computer

Interaction) on Human-Computer Interaction: Ergonomics and User Interfaces, Volume I - Volume I 829 - 833   - http://affect.media.mit.edu/pdfs/99.picard-hci.pdf

• Chapter one of Thrift, N. Non-Representational Theory: Space, Politics, Affect. London and New York, Routledge, 2008– See online journal version: http://nigelthrift.files.wordpress.com/2008/02/reinventing.pdf

• Various chapters in Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S. & Carey, T. Human-Computer Interaction. Wokingham, UK: Addison-Wesley, 1994

• Norman, D. Emotional design: Why we love (or hate) everyday things New York: Basic Books, 2004– See online version of chapter one: http://www.jnd.org/dn.mss/CH01.pdf– Prologue - http://www.jnd.org/dn.mss/CH00_Prolog.pdf– Epilogue - http://www.jnd.org/dn.mss/CH-Epilog.pdf

• Sampson T. D. (2011) “Contagion Theory Beyond the Microbe” Special Issue: In the Name of Security, Theory Beyond the Codes, Ctheory Journal (Archived here: http://www.ctheory.net/articles.aspx?id=675)

• McCarthy and Wright (2004), Technology as Experience, MIT Press, see chapter one “Living with Technology.” http://mitpress.mit.edu/sites/default/files/titles/content/9780262633550_sch_0001.pdf

Additional reading: Brennan, T. The Transmission of Affect, Ithaca & London: Cornell University Press, 2004

Key reading for prototyping • See chapter ten in Moggridge, B. Designing Interactions. Cambridge, MA: The MIT Press, 2007• Norman, D. The Design of Everyday Things. New York: Basic Books, 1988

Journals and Website Resources• Media and Culture Journal “Affect” 8/6 2005• http://journal.media-culture.org.au/0512/ Look at Eric Shouse’s article.

• Brun, A “Some Exploratory Notes on Produsers and Produsage”, snurb.info, http://snurb.info/index.php?q=node/329, accessed May 10, 2006

There is a number usability resources that you should make use of:• http://www.usabilitynet.org• www.useit.com/

Producer/Consumer Relations in the Age of Networks

Producer/Consumer Relations in the Age of Networks

or

The Politics (criticality) of User Testing

Tony D. Sampson

Presentation for MeCCSA Conference: National Media Museum Bradford Jan 09

What was the presentation about?

• Broadly – to think critically about the role of the “user” and user experience design.

• To follow up on various approaches to understand the role of design (UCD and UX) in developing relations between humans, and machines, and producers and consumers

• Specifically – look at eyetracking as a new technique in user testing

Eye tracking in user testing – measurement of attention and spontaneous and unconscious attraction of the

user

Video Examples of Eye Tracking Software

Look at first two

http://uk.youtube.com/watch?v=lo_a2cfBUGc&feature=related

http://www.youtube.com/watch?v=AEa8sFFU4a8&feature=related

http://uk.youtube.com/watch?v=PW4WrUeSoAY&feature=related

http://uk.youtube.com/watch?v=xKdOMgu0C5Q&NR=1

http://uk.youtube.com/watch?v=sVXjMXnU56E&feature=related

http://uk.youtube.com/watch?v=bZOVcmwHZZk&feature=related

New Media ResearchNew Media education as a convergence of

humanities and engineering (see The New Media Reader - MIT Press)

1. Software engineering = focus on how to make things work

– Early focus on ergonomic relation between human and machine

– Social factors, HCI and cognitive psychology

– User experience (affect, emotions at MIT)

2. Media theory tradition in humanities = critical exploration of social power relations in the new media = what are things all about

Practical concern: MS3305

– How to make two cultures coherently fit each other

UCD/UX: Literature

Literature Review…

– Mainly written by usability experts, design gurus & industrial consultants

• How user testing can improve the design of products

• Treats user as a consumer – customer experience

– Academic framework: HCI – hybrid of cognitive psychology, sociology and industrial design

• Interdisciplinary - How to focus the attention of the user on products

Critical Intervention

• Usability celebrates itself…

• Decidedly uncritical

‘Nobody is ever against usability’ Donald A. Norman Emotional Design New York:

Basic Books 2004 p. 39

The User/Producer Debate

• User/Programmer = Consumer/Produce (Galloway and Thacker, 2007)

The tragedy of interactive software

• User passive experience• Programmer “executes” the

user

User Generated Content (Web 2.0)

• Social factory producing consumer subjectivities See Immaterial Labour 2.0 Coté and Pybus, 2007 http://www.ephemeraweb.org/journal/7-1/7-1cote-pybus.pdf

The User

“…the consumer becomes producer as the public become participant role player”

McLuhan and Nevitt. Take Today: The Executive as Dropout. Ontario: Longman Canada, 1972 p. 4

The User

Toffler’s ‘prosumer’ economics” Alvin Toffler, 1971, Future Shock, Pan, London. See also Toffler, A. The Third Wave. London, Glasgow, Sydney, Auckland. Toronto, Johannesburg: William Collins, 1980 p. 27

The Produser (see Bruns 2006)

Bruns (2006) Towards Produsage: Futures for User-Led Content Production. In Sudweeks, Fay and Hrachovec, Herbert and Ess, Charles, Eds. Proceedings Cultural Attitudes towards Communication and Technology 2006, pages pp. 275-284, Tartu, Estonia.http://eprints.qut.edu.au/4863/1/4863_1.pdf

The User-Producer Debate (see Bruns 2006)

• Commercial sectors promotion of user participation in media production (Bruns, 2006)

• Harnessing the Hive

• Herz, J.C.: 2005, Harnessing the hive, In J. Hartley (ed.), Creative Industries, Blackwell, Malden, Mass., pp. 327-41.

• Customer-made Trendwatching.com, 2005a: Customer-made, available at http://www.trendwatching.com/trends/CUSTOMER-MADE.htm

• How users can make and customize products

• Role of user in production process• Captured in the electronics company

Philips’ identification of Lead Users: • ‘those consumers that face the needs that

will be general in the marketplace, but face them months or years ahead of the rest of the marketplace’

• Co-creation

New Economy

Critical Approach

1. Critique of the managerial aims of HCI to focus user attention and memory on software products (control of cognitive consumption)

2. Critique the move to capture the affective and emotional landscapes of users (control of noncognitive consumption)

3. Critique the role of experience research in establishing relations between consumers, products and brands

1. Goals of UsabilityFocusing User Attention

‘If we know that people are distracted, often involuntarily, how is it possible to get their attention again without allowing them to miss the “window of opportunity”’

(Preece et al p. 101)

Crary’s Thesis on AttentionAs a way of rethinking attention

• Capitalist production and management of…

• ‘The attentive subject’

• ‘through the… control of external procedures of stimulation as well as a wide-ranging technology of “attraction”’ (Crary p. 25)

‘The attentive subject’

• Traces the ‘disciplinary goals’ of communication products

• Implicit enforcement of attentiveness

• Modes of fixation

• Sedentary lifestyles

• Production of docile bodies always linked to intensified patterns of consumption

Technology of “attraction”’

• Eye tracking produces attentive subjects

Crary’s Sinister Turn• Attention posed as normative

and implicitly natural function

• Its impairment disrupts social cohesion

• Inattention and distraction are pathologized

• Inattention characterized by impulsiveness, short attention span, low frustration tolerance, distractibility, aggressiveness and… linked to feeling of underachievement (Crary pp. 35-36)

Emotional Design

2. Shift toward the capture of affective and emotional landscapes of users

Emotional Design

• Not just the measurement of behavioral use and reflection, but user testing of spontaneous and unconscious responses to products

• Norman’s addition of the visceral level of user interaction

• The ‘affective processing’ of the user

• The ‘automatic, prewired layer of human interaction’

(Norman pp. 22-23)

Emotional Designusers as consumers

• Emotional design occurs ‘in the world of products…’

• ‘Brands are all about emotions’

• They ‘draw the consumer towards the product’

• Emotional branding is about building relationships with users…

(Norman pp. 59-60)

3. The role of experience design in establishing [affective] relations between consumers, products and brands

New Media Producer/Consumer Relation Nigel Thrift (2008)

• Producers of commodities and brands establish passionate, affective relationship with consumers (p. 245).

• The corporate exploitation of noncognitive and pre-discursive realm of the user

Thrift

Corporations are in the business of making

‘hormonal splashes through increasing contact with consumers'

Attempts to manipulate the emotional mood of consumers

To affect the consumer’s senses

Consumer arousal

How are relations established?

1. ‘generation of engagement’

2. ‘generation of passions’

3. ‘new media hypnosis’

‘generation of engagement’

• Customizable Experiences enhances– Emotional

identification– Commitment to

brand

Customized Intelligence

Customer ProfilingUser Testing

The ‘generation of passions’

The added value of emotions and affects

Sensory design of commodities

Sensory Design

Scented laptops

Media

Hypnosis

New Media Hypnosis

• Contemporary somnambulist increasingly sinking into a ‘technological unconsciousness’ (Thrift, 2008)

New Media Hypnosis

• ‘Hypnosis involved a narrowing of attention, it paradoxically also enabled subjects to expand their awareness, in effect to see and remember more…(Crary p. 68)

The Somnambulist

New Media Hypnosis• Results of usability guru

Jacob Nielsen’s eye tracking research

• Users do not look at banner ads

• Proposes ‘unethical’ practice of hiding paid advertising in editorial content

• Indirect marketing, Trojan ads

• Not simply about attracting attention…

Heatmap from Nielson’s Useit.com

New Media Hypnosis

• Modern distraction is not a disruption of natural attention, but a constitutive element of the many attempts to produce attentiveness in human subjects (Crary p.

49),

The Somnambulist User

UXD at the heart of political power

ObamaUXD

“What makes for a successful use of web media in a Presidential campaign is not unlike what makes for any successful marketing enterprise: the user experience. User experience designers work hard to ensure that the a human’s interaction with technology is as pleasing as possible, but the irony is that the best user experiences are the ones that go unnoticed by the user — they just work. By that standard, the Obama user experience was a resounding success, enabling his supporters to feel like they were a part of the campaign.”

(Read the full interview here: Dream Jobs You’ve Never Heard of: Director of User Experience for Obama for America Campaign)

Photo by Jeff Ellis

Neuromarketing

• Further reading

• Brain Activity Measures Response to Ads, Commercials

• ‘Tarde as Media Theorist’: an interview with Tony D. Sampson, by Jussi Parikka

Seminar

• Read passages from Ctheory article

• Seminar Question: Why is absorption the ideal?

• What is the relation between neuromarketing and user experience design?

“Nothing... is less scientific than the establishment of this absolute separation, of this abrupt break, between the voluntary and the involuntary, between the conscious and unconscious. Do we not pass by insensible degrees from deliberate volition to almost mechanical habit?” [83]

Neuromarketing Over a hundred years later and Tarde's notion of the inseparability of voluntary and involuntary behavior is becoming central to biopolitical endeavors to organize consumptive

labor. Just as Thrift argues that the contemporary exercise of biopower evident in network science closely follows a Tardean trajectory, [84] the so-called neuromarketing expert claims to be able to measure the inseparable and anesthetized degrees between conscious and unconscious consumption. Drawing on recent inventions in neuroscience to inform such business enterprises, the neuromarketing expert claims to be able to gauge the spontaneous flows of consumer passion for services, brands and products. With ready access to advanced emotional recognition software and affective dataflows collected from the "user testing" of consumption experiences increasingly delivered online and through mobile devices, these highly qualified experts endeavor to prime environments for future purchase intent. Blending eye tracking software with electroencephalography (EEG) and galvanic skin response (GSR), companies like Berkeley based NeuroFocus not only measure a consumer's cognitive attention and memory retention, but claim to directly tap into what a consumer "feels about a product." [85] The combination of eye movement with the measurement of electrical activity in the brain, heart rate, and skin temperature to effectively record a user's emotional arousal during consumption, supplants the subjective inaccuracies of older marketing techniques of self-reporting, like questionnaires, surveys and focus groups.

Another innovation from the Danish company, iMotions, flags a distinct Tardean turn in market research technology. Distinct from slightly older methods that tended to measure either voluntary attention (bodily gestures, orientation, voice intonation, eye contact and evasion, and nervous responses) or involuntary inattention (increases in heart, pulse and breathing rates, and body temperature and sweating) the Emotion Tool claims to tap into the relation between the two. It targets, as such, the space in between the implicit, unconscious part of the brain (the limbic system), which is widely recognized as being hardwired to the nervous system and physical reactions, and the explicit, conscious system (the frontal cortex) associated with cognitive attention. It is the somatic memory, physical responses and emotions of the implicit system that are supposed to prime or guide the explicit system. [86] As the developer of the Emotion Tool claims:

It is now generally accepted that emotions dominate cognition, the mental process of the ability to think, reason and remember. Therefore, there is a rapidly increasing interest in methods that can tap into these mostly subconscious emotional processes, in order to gain knowledge and understanding of consumer behavior. [87]

The Emotion Tool tracks facial expressions, particularly those that occur around the eyes, the amount of blinking, the duration of the gaze, along with pupil dilation to measure emotional engagement. It further incorporates an algorithmic assessment of two dimensions of the emotional responses captured by the technology: emotional strength and affective valence. The first gauges the level of excitement an external stimulus provokes in the consumer, the second, measures the feelings that follow the stimulus -- the degree of attraction or aversion that an individual feels toward a specific object or event. Scores are calculated from a range of pleasant, unpleasant, or neither pleasant nor unpleasant. High scores are defined as "affective," low scores "unaffective."

Neuromarketing ushers in new methods of persuasion designed to sidestep the cognitive realm of visual representation and tap into the implicit, unconscious affective systems of consumption. Over and above focusing on what a consumer cognitively consumes in terms of visual attention (assumed to be atop of the Kantian hierarchy of the senses), neuromarketers measure the streams of affect the user somatically absorbs in the atmosphere. As the enthusiastic CEO of NeuroFocus puts it, a combination of techniques helps the marketer to go beyond conscious consumer engagement with a product and actively seek out what unconsciously attracts them.

Absorption is the ideal because it signifies that the consumer's brain has not only registered your marketing message or your creative content, but that the other centers of the brain that are involved with emotions and memory have been activated as well. The latest advances in neuroscience have revealed that all three of these key elements -- attention, emotion and memory retention -- are essential to the formation of what we call "persuasion"- which in turn means purchase intent. [88]

This inherently Tardean appeal to the indivisible neurological space between volition and mechanical habit suggests that "subliminal advertising," as Thrift notes, "does work." [89]

• [83] Gabriel Tarde, The Laws of Imitation, xi. • [84] Nigel Thrift, "Pass it On: Towards a Political Economy of Propensity," 24. • [85] Dr. A. K. Pradeep, "Persuasion: The Science and Methods of Neuromarketing." • [86] Jakob de Lemos, "Measuring Emotionally 'Fuelled' Marketing," Admap Magazine, Issue 482, April 2007, 40-42. • [87] Ibid. • [88] Dr. A. K. Pradeep, "Persuasion: The Science and Methods of Neuromarketing." • [89] Nigel Thrift, "Pass it On: Towards a Political Economy of Propensity," 22.