Real-Time Research

Mark Friedman Principal Technical Analyst

Analyzing Twitter Posts During Games-Learning-Society 2009

2

CTC Overview • 501(c)(3) nonprofit established in 1987

• Staff of 1,400+ professionals

• More than 50 locations

• 900,000 sq. ft., including labs & demonstration space

• Top 100 Government Contractor

• Quality/EH&S Management System comprised of industry-best models: ISO 9001 (Quality) and 14001 (Environmental), AS9100 (Aerospace), and CMMI-SE/SW (Systems/Software Engineering)

• Nationally recognized security capabilities with 300,000+ sq. ft. of Top Secret/Sensitive Compartmented Information Facility Space, JWICS, SIPRNet, and NIPRNet access

3

Why the Different Session Title?

Validating Learning Initiatives with Real Time Collaborative Research

Or what it is really is about – Real Time Research performed at GLS5 in Madison, WI during June 2009

4

Overview

• Participants in this session will

– Learn how to accomplish simple learning research

– Produce a quick-turnaround presentation

– Conduct complex research on games and learning

5 5

Real Time Research – Starting Point

• Theory card: Behaviorism • Topic: Social Networks • Method: Statistics

Star-studded facilitators included • Dr. Constance Steinkuehler • Dr. Kurt Squire • Dr. Eric Zimmerman

6

Research Cards – They Were Real

7

Research Cards • Received the three game cards

– “Behaviorism” as Theory

– “Social Networks” as Topic

– “Statistics” as Method

• Chose to investigate the nature of the tweet content tagged as #GLS & #GLS09

8

Topic Selection

• Research Question – Do Twitter posts during 24 hours (one conference day from 2pm until 2pm the next

day) refer to the self (“Me”), to another person’s speech or action (“You”), or both – a tweet of a community nature (“Us”).?

• In investigating the spirit of the social network content, we sought to

– Observe the type of messages participants exchanged during a professional conference

– Examine, and possibly challenge, the common perception that Twitter is a platform for excessive ego blasting, manifested in self-display

9

Methodology • A tweet expressing a personal action, thought, or intention was categorized as “Me”

• A tweet expressing another person’s action, thought, speech, or intention was categorized as “You”.

– In the #GLS and #GLS09 context, most of the “You” category consists of tweet content related to a session (or keynote) speaker.

• A tweet expressing a call to action to others (i.e. “who would like to play later in the arcade?” or an RT (Response Tweet)[1] was interpreted as a community-natured content and was categorized as “Us”.

• A tweet containing at least two of the above categories, or ambiguous content, which is disputed among group members was categorized as “Unidentified”.

[1] RT is a frequently-used one-click direct reply feature on Twitter.

10

Breakdown of the Data by Category

Category Raw Number

Percentage Definition

Me (self) 43 18% About the writer

You (other) 116 50% About someone else or event

Us (community)

69 29% About writer in a group

Unidentified 7 3% Unclassified

235 100%

11

Graph of the Data by Category

12

Full Statistics from Twitter • GLS Hashtag (~550)

– Wed 10 June 1247pm - Wed 10 June 737pm - about 100 (note the gap) – Thu 11 June 945am - Thu 11 June 459pm - about 300 (note the gap) – Fri 12 June 856am - Fri 12 June 1112am - about 100 – Fri 12 June 1113am - Fri 12 June 142pm - about 50

• GLS09 Hashtag (~540) – Wed 10 June 1133am - Wed 10 June 748pm - about 100 – Wed 10 June 748pm - Thu 11 June 1113am - about 150 – Thu 11 June 305pm - Thu 11 June 506pm - about 100 (note the gap) – Thu 11 June 507pm - Fri 12 June 814am - about 50 – Thu 11 June 812pm - Fri 12 June 1113pm - about 100 (note the overlap here) – Fri 12 June 1114am - Fri 12 June 201pm - about 40 (note the overlap here)

• IRC partial capture of GLS09 Hashtag – 631 Twitters total -- from 511pm Thursday June 11 to 1224pm Sunday June 14

13

Analysis – Trends Emerge! • Trends

– In the morning there were more tweets about the “self”.

• As the day progressed, and peaking in the evening and night, socialization messages increased in proportion, overall increasing the weight of the “Us” (community) category.

– “You” tweets were more prominent during conference sessions, especially during keynote sessions.

– As the GLS conference progressed, a community identity formed, resulting in more “Us” tweets .

• The second half of the day had an overall larger “Us” portion than the first part.

14

Analysis – Trends Emerge!

• A bonus trend: Self-reflective tweets demonstrating aesthetic caring about the Twitter platform

• “oops, sry for spam #gls09”

15

Three types of conclusions!

Initial Reflective Ongoing

16

Initial Conclusions • The “Me, You, Us” research is merely a drop in the sea of

possible investigations on social networks – observing people’s communication trends and analyzing comparative statistics

– Shows us that Twitter

• Has become an integral communication channel for professional conferences in general, and in particular in GLS 2009, where it was used for research, game play and analysis

• Functions as a real-time communication tool as originally designed

17

Initial Conclusions • The #GLS and #GLS09 tweets

– Enhanced the depth of discussion around games, learning, and society by allowing every writer to present their thoughts and challenge things presented officially on stage

• This type of liberation or democratization of professional communication

– Provides a platform to every participant (as well as those who could not make it to the conference, as seen in our “Us” example above)

– Reshapes the presenter-participant power hierarchy that exist in traditional conferences

18 18

Reflective Conclusions • As one conference-goer tweeted weeks after the event itself

“One thing we noticed at #gls09 – if your presentation couldn’t produce Twitter one liners, it did not exist.” (@cstubbs, July 29 2009, personal communication)

19

Ongoing Conclusions

• How might this type of social network-driven approach to event attendance affect professional conferences in the future?

Injecting Twitter into that conversation fundamentally changed the rules of engagement. It added a second layer of discussion and brought a wider audience into what would have been a private exchange. And it gave the event an afterlife on the Web. Yes, it was built entirely out of 140-character messages, but the sum total of those tweets added up to something truly substantive, like a suspension bridge made of pebbles.

How Twitter Will Change the Way We Live By Steven Johnson Friday, Jun. 05, 2009 (Time Magazine)

20

21

Questions

22

Technical Point of Contact:

Mark Friedman Principal Technical Analyst

757-788-9974 friedmam@ctc.com

Business Development Point of Contact:

David A. Kingston, P. E. Director, Learning and Human Performance Solutions

573-329-8548 kingstod@ctc.com