![Page 1: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/1.jpg)
Handicapping & Dark Horses
What Design Thinking Can Learn from Horse Racing
![Page 2: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/2.jpg)
![Page 3: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/3.jpg)
![Page 4: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/4.jpg)
![Page 5: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/5.jpg)
![Page 6: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/6.jpg)
2 Techniques
![Page 7: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/7.jpg)
#1 Handicapping
![Page 8: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/8.jpg)
handicapping:the art of evaluating a horse’s performance in order to determine the outcome of a race
![Page 9: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/9.jpg)
![Page 10: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/10.jpg)
![Page 11: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/11.jpg)
handicap the ideas
![Page 12: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/12.jpg)
![Page 13: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/13.jpg)
running a rapid experiment
![Page 14: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/14.jpg)
rapid experimentation:using prototypes to evoke behaviors that quickly tell us whether our ideas meet users’ needs
![Page 15: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/15.jpg)
rapid
experiments
usability test
survey
feedback methods
what will someoneactually do
what they might do
![Page 16: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/16.jpg)
rapid experiments separate what customers say from what they do in the real world
![Page 17: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/17.jpg)
what is an experiment?• a scientific procedure—a controlled empirical test of a
hypothesis
• hypotheses include (never null!):-A causes B-A is better, bigger, faster than A’-A changes B more when we do/provide X
• requirements:– independent variable that can be manipulated– dependent variable that can be measured– random assignment to condition (conservatively)
![Page 18: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/18.jpg)
![Page 19: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/19.jpg)
![Page 20: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/20.jpg)
+
![Page 21: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/21.jpg)
![Page 22: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/22.jpg)
What can design experiments test?• are people interested?
• will they do/use this given all other choices/demands available?
• does it meet the need you designed for?• does it have the desired (and not
undesired) effects?• can people figure out how to use/do it?• which design is better (at any of the
above)?
![Page 23: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/23.jpg)
How do you do this at home?1. Pick an idea 2. Make a list of all the questions you have
about the efficacy of the idea3. Select the most critical question to
success4. Generate a hypothesis5. Design an experiment to test your
hypothesis6. Create the prototype to support the
experiment
![Page 24: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/24.jpg)
Before you move into prototyping, make a plan:• What questions do you have about the
effectiveness of the ideas?• What is my hypothesis about what will happen?• What kinds of observations would validate my
hypothesis?• What prototype(s) do I need to create for these
observations?• How should I run this experiment?
Use the prototype to• Create an experience that evokes the desired
behaviors• Evaluate your working hypotheses
Don’t create the prototype before designing your experiment
![Page 25: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/25.jpg)
![Page 26: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/26.jpg)
Make a list of questions
Concepts we generate often have implicit assumptions about how people will respond and what people will doWhat are the crucial questions that could make or break the success of your idea?
![Page 27: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/27.jpg)
Select the most critical question
Will people be more likely to select a rental if it has a nice photo?Will homeowners be willing to let photographers photograph their space?How expensive is it create this service?Will the photos really look better?
![Page 28: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/28.jpg)
Generate a hypothesisA hypothesis is your prediction about how people will behave.
A lot more people will rent an apartment that has been professionally photographed than one that has not.
![Page 29: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/29.jpg)
Design an experiment• the goal is to evoke “real”
behaviors in “real” situations • the scenario must be
– believable– immersive– natural
• and, allow you to test/measure what you need to
• a word on “confederates”
![Page 30: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/30.jpg)
![Page 31: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/31.jpg)
Plan for measurement
people are notoriously bad at predicting (and also remembering/reporting) their own behaviors
![Page 32: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/32.jpg)
Plan for analysis• before conducting your experiment, figure out
how you’re going to analyze your data• what conclusions will you be able to draw from the
data?
normal photo
pro photo
4 5Did theyrent theapartment?
![Page 33: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/33.jpg)
Design your experiment– A/B Test – Show ½ the people the a rental with
normal photos, ½ the same rental with professional photos.
– Constrain to one geographic area– Select 2 popular and 2 unpopular
apartments.–Measurement: What percentage of people
who view the apartment rent it in condition A vs. B.
![Page 34: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/34.jpg)
Create the prototype
![Page 35: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/35.jpg)
run the experiment&
analyze your results
![Page 36: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/36.jpg)
Tip: You can follow up with qualitative interviewing
![Page 37: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/37.jpg)
![Page 38: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/38.jpg)
#1 Handicapping
![Page 39: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/39.jpg)
Dark Horse#2 Dark Horses
![Page 40: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/40.jpg)
dark horse:a contestant that seems unlikelyto succeed, but emerges to prominence against all odds
![Page 41: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/41.jpg)
dark horse idea1. dark: explores a space that is risky,
difficult, radical, or orthogonal
2. brainstormed after more traditional ideas
3. refined enough that can be prototyped and tested with a rapid experiment (can’t be infeasible)
![Page 42: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/42.jpg)
rational1. Designers need to preserve ambiguity to
leave themselves open to new ideas
2. Taking the dark horse idea to the point you can evaluate it keeps the design space from shrinking too fast
3. Even if the dark horse doesn’t work, you will gain insights that will make your final design better
![Page 43: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/43.jpg)
Example 1
Design the next generation cinema/broadcastingimaging system
![Page 44: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/44.jpg)
Original idea: Immersive movie watching experience where viewer’s head and eye position affect the perspective and focus of a sceneDark Horse: What if the camera was everywhere?Rigs of multiple Kinects and software capture 3D info of a scene for post capture scene manipulation.
![Page 45: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/45.jpg)
Example 2
Design a system to improve street navigation forpeople on vacation
![Page 46: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/46.jpg)
Original idea: Hotter/ColderPhone vibrates differently as you are getting closer or farther from the direction you should be going.Dark Horse: Stalker NavigationNo maps, crowdsourced directions, follow the stranger.
![Page 47: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/47.jpg)
Dark Horse#2 Dark Horses
![Page 48: Intro to Rapid Experimentation - Intel July 2015](https://reader030.vdocument.in/reader030/viewer/2022033022/58a31f561a28ab71398b492d/html5/thumbnails/48.jpg)
Handicapping & Dark Horses
What Design Thinking Can Learn from Horse Racing