CSCL 2011 | Keynote Augmented Social Cognition: How Social Computing is Changing eLearning

Ed H. Chi

Google Research Work done while at Palo Alto Research Center (PARC)

2008-05-13 CSCL 2011 Keynote

2008-05-13 CSCL 2011 Keynote

2008-05-13 CSCL 2011 Keynote

Prelude:  A  personal  learning  story  To:  chi@acm.org  From:  Brad  Barrish  <brad@…removed.for.privacy….com>  Subject:  Pancreatic  cancer  Date:  Thu,  1  Feb  2007  21:37:55  PST    Hey  Ed.  I'm  a  fellow  del.icio.us  user  and  noticed  you  bookmark  a  lot      of  pancreatic  cancer  stuff.  I'm  at  home  with  my  dad  who  was  diagnosed      a  little  over  a  year  ago  and  is  now  at  the  tale  end  of  things.  I've      learned  a  lot  through  his  treatments  and  about  what's  out  there.  I      dunno  if  it's  something  you  or  a  family  member  has,  but  just  wanted      to  drop  you  an  email.  Be  well.    Brad  

Talk  in  3  Acts  

n  Act  I:  The  Invisible  –  Social  Search  

n  Act  II:  The  Visible  –  Shared  Annotations  

n  Act  III:  The  Abstracted  –  Shared  Knowledge  Space  

2008-05-13 CSCL 2011 Keynote

The  Importance  of  Social  Signals  in  eLearning  

Act  I:    Invisible  Social  Signals  from  the  Crowd  

Joint  work  w/  Todd  Mytkowicz,  Rowan  Nairn,  Lawrence  Lee    [Chi  and  Mytkowicz,  Hypertext2008]  [Kammerer  et  al.,  CHI2009]    

2008-05-13 CSCL 2011 Keynote

Using  Information  Theory  to  Model  Social  Tagging  [Ed  H.  Chi,  Todd  Mytkowicz,  ACM  Hypertext  2008]  

Topics  Concepts  

Users   Documents  

Tags  T1…Tn  

Encoding  Decoding  

Noise  

2008-05-13 CSCL 2011 Keynote

Tagging  Behavior  

H(Tag)  shows  tag  saturation   H(Doc  |  Tag),  browsability  

2008-05-13 CSCL 2011 Keynote

Implication  

I(Doc;  Tag)    Mutual  Information   Raise  in  avg.  tag  /  bookmark  

2008-05-13 CSCL 2011 Keynote

TagSearch:  MapReduce  Implementation  

n  Spreading  Activation  in  a  bi-­‐graph  n  Computation  over  a  very  large  data  set  

–  150  Million+  bookmarks  

Tags URLs

P(URL|Tag)

P(Tag|URL)

2008-05-13 CSCL 2011 Keynote

2008-05-13

Guide

Web

Howto

Tips Help

Tools

Tip

Tricks

Tutorial

Tutorials

Reference

Semantic Similarity Graph

TagSearch:  Use  Semantic  Analysis  to  Reduce  Noise          http://mrtaggy.com    

CSCL 2011 Keynote

2008-05-13 CSCL 2011 Keynote

Experiment  Design    [Kammerer  et  al.  CHI2009]  

n  2  interface  x  3  task  domain  design  –  2  Interface  (between-­‐subjects)  

n  Exploratory  vs.  Baseline  –  3  task  domains  (within-­‐subjects)  

n  Future  Architecture,  Global  Warming,  Web  Mashups  

n  30  Subjects  (22  male,  8  female)  –  Intermediate  or  advanced  computer  and  web  search  skills  –  Half  assigned  Exploratory,  half  Baseline.  

n  For  each  domain,  single  block  with  3  task  types:  –  Easy  and  Difficult  Page  Collection  Task  [6min  each]  –  Summarization  Task  [12min]  –  Keyword  Generation  Task  [2min]  

2008-05-13 CSCL 2011 Keynote

Evauation  Results  [Kammerer  et  al.,  CHI2009]  n  Exploratory  interface  users:  

–  performed  more  queries,    –  took  more  time,    –  wrote  better  summaries  (in  2/3  domains),    –  generated  more  relevant  keywords  (in  2/3  domains),  and  –  had  a  higher  cognitive  load.  

n  Suggestive  of  deeper  engagement  and  better  learning.  n  Some  evidence  of  scaffolding  for  novices  in  the  keyword  

generation  and  summarization  tasks.  

2008-05-13 CSCL 2011 Keynote

Act  II:  Visible  Social  Signals  from    Shared  Highlighting  

   

Kudos  to  Lichan  Hong,  Les  Nelson    

[Hong  et  al,  AVI2008]  [Nelson  et  al.,  HCII  2009]  

2008-05-13 CSCL 2011 Keynote

Finding  a  Restaurant  

n  Appropriate  for  the  occasion  

2008-05-13 CSCL 2011 Keynote

Poor heuristic

Good heuristic

Heuristics  

2008-05-13 CSCL 2011 Keynote

“Hints”  

Solo

Cooperative (“good hints”)

2008-05-13 CSCL 2011 Keynote

SparTag.us:  Social  Highlighting  

2008-05-13 CSCL 2011 Keynote

SparTag.us:  Social  Highlighting  

n  In  situ  tagging  while  reading  n  Highlighting  n  Shared  notebooking    n  Sharing!  

2008-05-13 CSCL 2011 Keynote

Highlighting  as  Importance    Indicator  

recall

first-visit

2008-05-13 CSCL 2011 Keynote

n  Sensemaking  task  –  Find  and  read  material  about  “Enterprise  2.0  mashups”  in  order  to  

write  two  essays  

n  Seeds:  “expert”  content  for  scaffolding  –  Tags  from  del.icio.us  –  URLs  from  Google/PageRank  –  Constructed  and  then  shared  through  social  mechanisms  (i.e.,  a  

SparTag.us  “friend”)  

n  Performance  Measures  –  Learning  gain:  Pre/Post  Knowledge  Test  

Evaluation  Task  &  Metric  [Nelson  et  al.,  HCII2009]  

scorePretest - scoreMax scorePretest -scorePosttest

=Gain

2008-05-13 CSCL 2011 Keynote

Procedure  

Demographics  &  Pretest  

SF SparTag.us  with  ‘Friend’  

SO SparTag.us  Only  

WS Without  SparTag.us  

Posttest  

2008-05-13 CSCL 2011 Keynote

Results:  Learning  Gain  N=18    SparTag.us  +  Friend  superior  to  both  individual  conditions  No  difference  between  the  two  control  conditions  

2008-05-13 CSCL 2011 Keynote

URL Kind Code Blog B Conference C Employment E MySpartagus M News N OpenSource O Search S Vendor V Wikipedia W Consultant X

Observation  URL  Kind   Code  Blog   B  Conference   C  Employment   E  My.Spartag.us   M  News   N  OpenSource   O  Search   S  Vendor   V  Wikipedia   W  Consultant   X  

2008-05-13 CSCL 2011 Keynote

Von  Restorff  Isolation  Effect  [1933]  n  As  applied  to  highlights,  the  von  Restorff  isolation  effect  

suggests  that  readers:  n  (a)  tend  to  focus  on  and    n  (b)  learn  what  is  marked,    n  whether  the  information  is  important  or  not.  

–  Nist  and  Hogrebe  87  

2008-05-13 CSCL 2011 Keynote

Act  III:    Abstracted  Knowledge:  The  Science  of  Understanding  Wikipedia  

Kudos  to  Bongwon  Suh,  Niki  Kittur    [Kittur  et  al.,  CHI2007]  [Suh  et  al.,  WikiSym  2009]    

2008-05-13 CSCL 2011 Keynote

Exponential  Growth  of  Wikipedia:  an  accepted  ‘fact’  

Number of Articles (Log Scale)

http://en.wikipedia.org/wiki/Wikipedia:Modelling_Wikipedia’s_growth

2008-05-13 CSCL 2011 Keynote

Growth  of  Edits  

2008-05-13 CSCL 2011 Keynote

Something  happened  in  early  2007  

2008-05-13 CSCL 2011 Keynote

Growth  of  Active  Editors  *In thousands

2008-05-13 CSCL 2011 Keynote

Slowing  Growth  in  Global  Activity  *In thousands

2008-05-13 CSCL 2011 Keynote

Earlier  Exponential  Growth  Model  n  Preferential  Attachment:  Edits  beget  edits  

–  more  number  of  previous  edits,  more  number  of  new  edits  

Growth rate of population

Current population

Growth rate depends on: N = current population r = growth rate of the population

2008-05-13 CSCL 2011 Keynote

!

dNdt

= r " N

!

N(t) = N0 " ert

Logistic  Growth  Model  n  Ecological  population  growth  model  

–  Also  depend  on  environmental  conditions  –  K,  carrying  capacity  (due  to  resource  limitation)  

!

dNdt

= rN(1" NK)

2008-05-13 CSCL 2011 Keynote

Match  to  Data:  #  of  New  Articles  n  Follows  a  logistic  growth  curve  

New Article

2008-05-13 CSCL 2011 Keynote

Struggle  for  Existence  -­‐  Darwin  n  Biological  system  

–  Competition  increases  as  population  hit  the  limits  of  the  ecology  

–  Advantage  go  to  members  of  the  population  that  have  competitive  dominance  over  others  

n  Analogy  –  Limited  opportunities  to  make  

novel  contributions  –  Increased  patterns  of  conflict  and  

dominance    

2008-05-13 CSCL 2011 Keynote

“Showering”  Hypothesis  What  drives  contributions  to  Wikipedia?  Cooperation  is  not  the  main  driver?  n  Hypothesis:  Conflicts  drives  most  of  the  contributions.  

–  How  do  we  measure  conflicts?  

n  Conflicts  cause  coordination  costs  to  go  up.  –  How  to  measure  coordination  costs?  

n  “negotiation  is  critical  to  helping  multiple  perspectives  to  converge  on  shared  knowledge.”    –  Stahl,  Group  Cognition,  Ch8,  2004  

2008-05-13 CSCL 2011 Keynote

2008-05-13

Conflict/Coordination  Effects  in  Wikipedia  (Kittur, Suh, Pendleton, Chi, CHI2007)

CSCL 2011 Keynote

Ratio  of  Reverted  Contributions    

Monthly Ratio of Reverted Edits

2008-05-13 CSCL 2011 Keynote

Visual  Analytics  over  Wikipedia  data  Mediator  Pattern  -­‐  Terri  Schiavo    [Suh,  et  al.,  VAST2007]  

Mediators

Sympathetic to parents

Sympathetic to husband

Anonymous (vandals/spammers)

2008-05-13 CSCL 2011 Keynote

WikiDashboard.com  

2008-05-13 CSCL 2011 Keynote

Coda:  A  Challenge:  A  modified  logistic  model  n  Carrying  Capacity  as  a  function  of  time.  

2008-05-13 CSCL 2011 Keynote

What  Did  We  Learn?  n  The  Common  Thread:  

–  Utilization  of  Social  Signals  for  Learning  and  Information  Access  –  Whether  it  is  invisible,  visible,  and  abstracted.  

n  The  Establishment  of  Common  Ground  –  Implicit  Coordination  –  Explicit  Coordination  –  Negotiation  

n  “All  collective  actions  are  built  on  common  ground  and  its  accumulation.”  –  Clark  and  Brennan,  1991  

2008-05-13 CSCL 2011 Keynote

Research  Vision  

Augmented  Social  Cognition  n  Cognition:  the  ability  to  remember,  think,  and  reason;  the  faculty  of  

knowing.  n  Social  Cognition:  the  ability  of  a  group  to  remember,  think,  and  

reason;  the  construction  of  knowledge  structures  by  a  group.  –  (not  quite  the  same  as  in  the  branch  of  psychology  that  studies  the  

cognitive  processes  involved  in  social  interaction,  though  included)  

n  Augmented  Social  Cognition:  Supported  by  systems,  the  enhancement    of  the  ability  of  a  group  to  remember,  think,  and  reason;  the  system-­‐supported  construction  of  knowledge  structures  by  a  group.    

Citation:  Chi,  IEEE  Computer,  Sept  2008  

2008-05-13 CSCL 2011 Keynote

From  Rote  Learning  to  Interaction  

2008-05-13 CSCL 2011 Keynote

Thank  you!  n  chi@acm.org  n  http://edchi.net  

2008-05-13 CSCL 2011 Keynote

What  I  will  not  talk  about  …  n  Motivation  

–  Cultural  and  economic  incentives  –  Personal  and  societal  values  –  Psychology  (e.g.  cognitive,  personality,  social)  

n  Policy  and  Investment  –  Resources  –  Teacher  training  –  Technological  investment  

n  With  the  Assumption  of  Motivation  and  Resources,  how  to  make  information  universally  accessible  and  useful  in  a  Web2.0  world?  

2008-05-13 CSCL 2011 Keynote

2008-05-13 CSCL 2011 Keynote

Lowering  Participation  /  Interaction  Costs  

n  Interaction  costs  determine  number  of  people  who  participate  

n  Surplus  of  attention  &  motivation  at  small  transaction  costs  

n  Therefore…  n  Important  to  keep  

interaction  costs  low  

Cost of participation #

Peop

le w

illin

g to

pro

duce

for “

free

”

2008-05-13 CSCL 2011 Keynote

2008-05-13 CSCL 2011 Keynote

Using  Machine  Learning  to  Detect  Conflicts  n  Counting  ‘Controversial’  labels  n  5x  cross-­‐validation,  R2  =  0.897  

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

Predicted controversial revisions

Actu

al c

ontr

over

sial r

evisi

ons

Collaborative  Knowledge  Building  n  “They  cannot  even  begin  to  coordinate  on  content  

without  assuming  a  vast  amount  of  shared  information  or  common  ground….  And  to  coordinate  on  process,  they  need  to  update  their  common  ground  moment  by  moment.  All  collective  actions  are  built  on  common  ground  and  its  accumulation.”  –  Clark  and  Brennan,  1991  

n  At  Web-­‐scale  social  learning,  what  we  know  about  the  nature  of  conflict  and  negotiation  is  woefully  inadequate.  

2008-05-13 CSCL 2011 Keynote

Google  Plus  as  a  Research  Platform  

2008-05-13 CSCL 2011 Keynote