michael edson @ j. boye 2011: jedi mind tricks for measuring and optimizing lofty goals (updated...
DESCRIPTION
Updated version of a talk first given at the e-Metrics Marketing and Optimization Summit, Washington, D.C., October 5, 2010. This version is for the J. Boye conference, Philadelphia, PA, 5-4-2011. The talk spells out the lofty goals of the Smithsonian Institution and the Smithsonian Commons project, and then describes some of the unconventional measurement methodologies we're using to figure out what to do and how to do it. J. Boye Conference page: http://jboye.com/conferences/philadelphia11/program/speakers/michael-edson/#presentationTRANSCRIPT
Michael EdsonDirector, Web and New Media Strategy
Smithsonian Institution, Office of the [email protected] | @mpedson
Jedi Mind Tricks for
OptimizingLofty Goals
andMeasuring
Preamble
Michael EdsonDirector, Web and New Media Strategy
Smithsonian Institution, Office of the [email protected] | @mpedson
@mpedson
Preamble
Michael EdsonDirector, Web and New Media Strategy
Smithsonian Institution, Office of the [email protected] | @mpedson
@mpedsonslideshare.net/edsonm
smithsonian-webstrategy.wikispaces.com
Preamble
Michael EdsonDirector, Web and New Media Strategy
Smithsonian Institution, Office of the [email protected] | @mpedson
@mpedsonslideshare.net/edsonm
smithsonian-webstrategy.wikispaces.com
Get the full story here
Preamble
Michael EdsonDirector, Web and New Media Strategy
Smithsonian Institution, Office of the [email protected] | @mpedson
@mpedsonslideshare.net/edsonm
smithsonian-webstrategy.wikispaces.com
…and here
Preamble
Michael EdsonDirector, Web and New Media Strategy
Smithsonian Institution, Office of the [email protected] | @mpedson
The Smithsonian Commonswww.si.edu/commons
Preamble
Michael EdsonDirector, Web and New Media Strategy
Smithsonian Institution, Office of the [email protected] | @mpedson
He’s not a spokesman!
First, we need to tell the story…
Smithsonian Strategic Plan
Smithsonian Secretary G. Wayne Clough
http://www.si.edu/about/
Four Grand Challenges
Smithsonian Strategic Plan
1. Unlocking the Mysteries of the Universe
2. Understanding and Sustaining a Biodiverse Planet
3. Valuing World Cultures4. Understanding the
American Experience
Four Grand Challenges
http://www.si.edu/about/
Smithsonian Strategic Plan
1. Unlocking the Mysteries of the Universe
2. Understanding and Sustaining a Biodiverse Planet
3. Valuing World Cultures4. Understanding the
American Experience
Four Grand Challenges
http://www.si.edu/about/
Smithsonian Strategic Plan
1. Unlocking the Mysteries of the Universe
2. Understanding and Sustaining a Biodiverse Planet
3. Valuing World Cultures4. Understanding the
American Experience
Four Grand Challenges
http://www.si.edu/about/
Smithsonian Strategic Plan
1. Unlocking the Mysteries of the Universe
2. Understanding and Sustaining a Biodiverse Planet
3. Valuing World Cultures4. Understanding the
American Experience
Four Grand Challenges
http://www.si.edu/about/
Smithsonian Strategic Plan
1. Unlocking the Mysteries of the Universe
2. Understanding and Sustaining a Biodiverse Planet
3. Valuing World Cultures4. Understanding the
American Experience
Four Grand ChallengesStrategy
Schmatergy! We’ve seen strategy
before!
Smithsonian Strategic Plan
1. Unlocking the Mysteries of the Universe
2. Understanding and Sustaining a Biodiverse Planet
3. Valuing World Cultures4. Understanding the
American Experience
Four Grand ChallengesNo! This is about solving big hairy
problems— “work that matters”
(via @timoreilly)http://radar.oreilly.com/2009/01/work-on-stuff-that-matters-fir.html
c
c
This is big, audacious, important stuff.
This is work!
This is relevance earned through a job well done.
First order questions
First order questions
• Where is this work going to take place? • What kind of organization, infrastructure,
platforms will be needed to support it?• What is the organizational change model?
How will change happen? • Who will be the innovators? The connectors?
The drivers of change?• …and how do you measure it?
Web andNew Media Strategy
Web andNew Media Strategy
http://smithsonian-webstrategy.wikispaces.com
More in Fast, Open, and Transparent: developing the Smithsonian’s Web and New Media Strategy http://www.slideshare.net/edsonm/michael-edson-the-smithsonian-web-and-new-media-strategy-what-it-is-how-we-made-it-and-why-it-makes-a-difference-3656578 And http://www.archimuse.com/mw2010/papers/edson/edson.html
http://smithsonian-webstrategy.wikispaces.com/
The strategy talks about an updated digital experience, a new learning model that helps people with their "lifelong learning journeys," and the creation of a Smithsonian Commons—a new part of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities.
http://smithsonian-webstrategy.wikispaces.com/
The strategy talks about an updated digital experience, a new learning model that helps people with their "lifelong learning journeys," and the creation of a Smithsonian Commons—a new part of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities.
http://smithsonian-webstrategy.wikispaces.com/
The strategy talks about an updated digital experience, a new learning model that helps people with their "lifelong learning journeys," and the creation of a Smithsonian Commons—a new part of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities.
http://smithsonian-webstrategy.wikispaces.com/
The strategy talks about an updated digital experience, a new learning model that helps people with their "lifelong learning journeys," and the creation of a Smithsonian Commons—a new part of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities.
Old Learning ModelNew Learning Model
http://smithsonian-webstrategy.wikispaces.com/
Balancing autonomy and control within the Smithsonian.rt of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities.
http://smithsonian-webstrategy.wikispaces.com/
Balancing autonomy and control within the Smithsonian.rt of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities.
Innovation at the Edges---
A commons in the middle
http://smithsonian-webstrategy.wikispaces.com/
…and the creation of a Smithsonian Commons—a new part of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities.
http://smithsonian-webstrategy.wikispaces.com/
…and the creation of a Smithsonian Commons—a new part of our digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities.
What is a Commons?
What is a Commons?
A set of resources maintained in the public sphere for the use and benefit of everyone
What is a Commons?
What is a Commons?
The Anti-Commons…
What is a Commons?
An organized workshop where raw materials can be found and assembled into new things.
What is a Commons?
Imagining a Smithsonian Commons• http://www.slideshare.net/edsonm/cil-2009-michael-edson-text-version
Museum Commons: A professional interactionhttp://www.slideshare.net/edsonm/museum-commons-a-professional-interaction-museums-and-the-web-2010-michael-edson-and-rich-cherry
What is a Commons?
What is a Commons?
Imagining a Smithsonian Commons• http://www.slideshare.net/edsonm/cil-2009-michael-edson-text-version
Museum Commons: A professional interactionhttp://www.slideshare.net/edsonm/museum-commons-a-professional-interaction-museums-and-the-web-2010-michael-edson-and-rich-cherry
What is a Commons?
Also…
A 15-minute talk at the Walker Art CenterText/footnotes: http://www.slideshare.net/edsonm/m-4402558
Video (starts at minute 12): http://channel.walkerart.org/play/opening-the-field/
Updated 6/21/2010
So now we’re at a moment…
The Grand Challenges
The Smithsonian Commons
How do you talk about these things in measurable business
terms?
So now we’re at a moment…
The Grand Challenges
The Smithsonian Commons
How do you use “information” to persuade, guide,
inform?
So now we’re at a moment…
The Grand Challenges
The Smithsonian Commons
How do you measure things that are…
difficult to measure?
Jedi Mind Tricks for
OptimizingLofty Goals
andMeasuring
Jedi Mind Trick #1Show your users to management
Relevance
http://www.youtube.com/watch?v=N5x4Sga0d1s
Q: Have you ever visited a Smithsonian Web site?
Jedi Mind Trick #2Use Organic Search and Alexa
Relevance
Unexpected Rivals in Google Search
Google Images
Wikipedia
Ocean.com
Discoveryeducation.com
NASA
Enchantedlearning.com
Relevance
Unexpected Rivals in Reach
Enchantedlearning.com
si.edu
discoveryeducation.com
ocean.com
Google ImagesWikipedia
Ocean.com…so much more reach than SI
that we don’t even show up on the graph…
Relevance
Unexpected Rivals in Reach (July – Sept, 2009)
Enchantedlearning.com
si.edu
discoveryeducation.com
ocean.com
Relevance
Unexpected Rivals in Reach (July – Sept, 2009)
Enchantedlearning.com
si.edu
discoveryeducation.com
ocean.com
Enchantedlearning.com is a two person team,
with more online reach than the world’s largest museum and research
complex!
Relevance
Traffic Trending Down
si.edu – 4% reach Wikipedia.org + 8% reachMoMA.org + 12% reachnpr.org + 20% reach
Jedi Mind Trick #3Humble yourself with
brandtags.net
Relevance
Brand Identity
Brandtags.netWe are the 560th of 928 brands
Jedi Mind Trick #4Highlight your competitors
Relevance
We’re competing with… everybody!
Relevance
We’re competing with… everybody!
Relevance
We’re competing with… everybody!
Jedi Mind Trick #5Use 3rd party research
Relevance
The Demographic Tsunami
Ages 12-17
Ages 18-29
Ages 30-38
Ages 39-48
Ages 49-60
Ages 61-69
Ages 70+0
10
20
30
40
50
60
70
80
90Online Content Creation by Age
Internet users
Per
cen
tag
e
November 2007 data: Pew Internet and American Life Project
Relevance
The Demographic Tsunami“Everything we hear from people we interview is
that today’s consumers draw no distinctions between an organization’s Web site and their traditional bricks-and-mortar presence: both must be excellent for either to be excellent.”
Lee RainiePew Internet and American Life Project
Jedi Mind Trick #6Demonstrable feedback from users
1,144 positive votes/comments on prototype (out of 1,171 total votes)
1,144 positive votes/comments on prototype (out of 1,171 total votes)
It’s just Google Spreadsheet hooked up
to a form.
Jedi Mind Trick #7Show validation from VIP’s
Jedi Mind Trick #8Measure end-user satisfaction
Jedi Mind Trick #9Bake measurement into the
platform from the start
Jedi Mind Trick #9Bake measurement into the
platform from the startSmithsonian Office of the
CIOis starting to study the
as-is measurement/analysisprogram
YESNO17%
83%
77
Survey HighlightsWhich web analytics tools are you using?
What are the some of the most valuable behaviors a user can perform on your site?
10%
29%
61%
Are there web-user activities NOT being measured
that you WANT to measure?
http://smithsonian-webstrategy.wikispaces.com/
The importance of measurement/analysis
and UX is laced throughout the
strategy
Project Charter (draft)
http://smithsonian-webstrategy.wikispaces.com/Smithsonian+Commons+Project+Charter
Project Charter (draft)
http://smithsonian-webstrategy.wikispaces.com/Smithsonian+Commons+Project+Charter
Measurement and analysis resources in the
project plan from the very start
Project Charter (draft)
http://smithsonian-webstrategy.wikispaces.com/Smithsonian+Commons+Project+Charter
Measurement and analysis resources in the
project plan from the very start
Eight measurable goals described
Project Charter (draft)
This is what the charter says…
Project Charter (draft)
3. Measurable Project GoalsThe goal of the Smithsonian Commons project is to stimulate learning, creation, and innovation through open access to Smithsonian research, collections and communities.
Project Charter (draft)
3. Measurable Project GoalsPrecise, measurable goals are a hallmark of successful project governance, but these kinds of outcomes are notoriously hard to define and measure in practice: seeking to define them too precisely or in too nuanced a way early in the project can be as counterproductive as not measuring them at all.
Project Charter (draft)
3. Measurable Project GoalsPrecise, measurable goals are a hallmark of successful project governance, but these kinds of outcomes are notoriously hard to define and measure in practice: seeking to define them too precisely or in too nuanced a way early in the project can be as counterproductive as not measuring them at all.
Project Charter (draft)
3. Measurable Project GoalsTherefore, the project will initially focus on three measurements: the amount of information in the Smithsonian Commons, the amount of activity on the Smithsonian Commons, and the satisfaction of people who use the Smithsonian Commons.
Project Charter (draft)
3. Measurable Project GoalsTherefore, the project will initially focus on three measurements: the amount of information in the Smithsonian Commons, the amount of activity on the Smithsonian Commons, and the satisfaction of people who use the Smithsonian Commons.
Project Charter (draft)
3. Measurable Project GoalsTherefore, the project will initially focus on three measurements: the amount of information in the Smithsonian Commons, the amount of activity on the Smithsonian Commons, and the satisfaction of people who use the Smithsonian Commons.
Project Charter (draft)
3. Measurable Project GoalsTherefore, the project will initially focus on three measurements: the amount of information in the Smithsonian Commons, the amount of activity on the Smithsonian Commons, and the satisfaction of people who use the Smithsonian Commons.
Project Charter (draft)
The specific, measurable 5-year goals1) 100 million items in the commons…
Project Charter (draft)
The specific, measurable 5-year goals1) 100 million items in the commons…2) 100 million transactions a year…
Project Charter (draft)
The specific, measurable 5-year goals1) 100 million items in the commons…2) 100 million transactions a year…3) High end-user satisfaction, both internally and internally, as measured through the Foresee Results measurement tool or equivalent
Project Charter (draft)
The specific, measurable 5-year goals1) 100 million items in the commons…2) 100 million transactions a year…3) High end-user satisfaction, both internally and internally, as measured through the Foresee Results measurement tool or equivalent
“Bits in: bits out”
(via Brewster Kahle)
Project Charter (draft)
Five additional measurable goals guide the project. The Smithsonian Commons will,4) Increase reuse of Smithsonian resources5) Be family-safe and school-safe6) Demonstrate new revenue models7) Drive traffic to unit websites8) Help eliminate cost and work at the unit level
Jedi Mind Trick #10Measurement is only one piece of
the equation
Jedi Mind Trick #10Measurement is only one piece of
the equation
It’s about understanding and improving
user experience
Jedi Mind Trick #10Measurement is only one piece of
the equation
You need a team
Smithsonian Commons Prototype
So let’s see what that’s like!
Thanks!!!
Michael EdsonDirector, Web and New Media Strategy
Smithsonian Institution, Office of the [email protected] | @mpedson