how the science of teams can inform team science nancy j. cooke march 13, 2015 team science retreat...
TRANSCRIPT
How the Science of Teams How the Science of Teams Can Inform Team ScienceCan Inform Team Science
How the Science of Teams How the Science of Teams Can Inform Team ScienceCan Inform Team Science
Nancy J. Cooke
March 13, 2015Team Science Retreat
Wake Forest School of Medicine ofWake Forest Baptist Medical Center
Overview• Why Team Science?• Update on NRC Study • My research and experience • A Multi-Level Systems Perspective
Micro Level: Challenges and SupportMeso Level: Challenges and SupportMacro Level: Challenges and Support
• Conclusion
Why Team Science?• Today’s problems require a team of multidisciplinary
individuals• Team Science is impactful (highly cited; Wuchty, et
al., 2007; Uzzi, et al., 2013)• Team Science is innovative (Uzzi, 2013)• Team Science is productive (Hall, et al., 2012)• Team Science has broad reach/uptake (Stipelman, et
al, 2014)
Enhancing the Effectiveness of Team Science: Symposium at ICPS
March 13, 2015
Board on Behavioral, Cognitive, and Sensory SciencesDivision of Behavioral and Social Sciences and EducationNational Research Council
An Update on the NRC Study of Team Science
6
Study Background• Rationale: Clear need to provide research-based
guidance to improve the processes and outcomes of team science
• Sponsors: NSF, Computer and Information Systems and Engineering Directorate and Elsevier
• Goal: Enhance the effectiveness of collaborative research in science teams, research centers, and institutes.
• Audiences: NSF and other public and private research funders; the scientific community; the SciTS community; universities; research centers and institutes.
Committee ChargeConduct a consensus study on the science of team science to recommend opportunities to enhance the effectiveness of collaborative research in science teams, research centers, and institutes… Explore: •How individual factors influence team dynamics, effectiveness and productivity•Factors at the team, center, or institute level that influence effectiveness •Different management approaches and leadership styles that influence effectiveness •How tenure and promotion policies acknowledge academic researchers who join teams•Organizational factors that influence the effectiveness of science teams (e.g., human resource policies, cyberinfrastructure)•Organizational structures, policies and practices to promote effective teams
Committee• NANCY J. COOKE (Chair), Arizona State University• ROGER D. BLANDFORD (NAS), Stanford University• JONATHON N. CUMMINGS, Duke University • STEPHEN M. FIORE, University of Central Florida• KARA L. HALL, National Cancer Institute• JAMES S. JACKSON (IOM), University of Michigan• JOHN L. KING, University of Michigan• STEVEN W. J. KOZLOWSKI, Michigan State University• JUDITH S. OLSON, University of California, Irvine• JEREMY A. SABLOFF (NAS), Santa Fe Institute • DANIEL S. STOKOLS, University of California, Irvine• BRIAN UZZI, Northwestern University• HANNAH VALANTINE, National Institutes of Health
Study StatusReport expected in AprilMore information is available at:http://sites.nationalacademies.org/DBASSE/BBCSS/CurrentProjects/DBASSE_080231
Research Base for Informing Team Science
• SciTS – Science of Team Science (itself a multidisciplinary approach)
• Social Science• Complex Systems• Communications• Management• Medicine• Physical Sciences
The Foresight Initiative• National Geospatial-Intelligence Agency (NGA)
has awarded Arizona State University a grant of $20 million
• Five-year partnership known as the Foresight Initiative will examine how climate change affects resources and contributes to political unrest, as well as articulate sustainability and resilience strategies.
Foresight: A Science TeamApproximately 60 Investigators• 15 ASU Faculty from 8 ASU units• Post docs, research faculty, graduate students• Three National Labs• National Geospatial Intelligence Agency• Expertise in visualization, modeling climate
change, cognitive science, social media, human factors
Foresight: Team Science is Challenging
• Communicating across disciplines• Role confusion• Meetings• Remote participation• Goal conflicts• Sub-teams• Authorship• Resource Allocation
My Research and Experience Relevant to Team Science
Team = Heterogeneous and interdependent group of individuals
(human or synthetic) who plan, decide, perceive, design, solve problems, and act
as an integrated system (vs. group)
Cognitive activity at the team level=
Team Cognition
Improved team cognition Improved team/system effectiveness
Heterogeneous = differing backgrounds, differing perspectives on situation
(surgery, basketball)
Teams and Cognitive Tasks
I’ve Studied Team Cognition in These Tasks
Uninhabited Aerial Vehicle Command and ControlNaval Mission Planning
Cyber DefenseIntelligence Analysis
Human-Underwater Robot InteractionMedical Emergency Teams
Professional CookingHuman-Robot Search and Rescue
Methods: Synthetic Task EnvironmentsA compromise between field studies and laboratory experiments
16
Uninhabited Aerial Vehicle – Synthetic Task
Environment
MacroCog
Underwater Robots
CyberCog
Teams LearnAs teams acquire experience, performance improves, interactions improve,
but not individual or collective knowledge
0
100
200
300
400
500
600
1 2 3 4 5 6 7 8 9 10
Mission
Team
Per
form
ance
Tm 1
Tm 2
Tm 3
Tm 4
Tm 5
Tm 6
Tm 7
Tm 8
Tm 9
Tm 10
Tm 11
• Individuals are trained to criterion prior to M1• Asymptotic team performance after four 40-min missions (robust finding)• Knowledge changes tend to occur in early learning (M1) and stabilize• Process improves and communication becomes more standard over time
40-min missionsSpring Break
Teams ForgetTeam forgetting is best predicted by interaction based measures, not
by individual forgetting (despite shared score components)
Regression model made up of individual decrements: F (4, 20) = 2.018, MSe=5880.23, p>.10, R2 = .29
Introduction of coordination and team SA: F (10, 14) = 2.71, MSe = 4011.88, p < .05, R2 = .66
8-10
wee
k re
tenti
on in
terv
al
Membership Matters• 117 males(92) & females(25) divided into 39
3-person (unfamiliar) Session 2 teams• Two between subjects conditions (retention
interval and familiarity) randomly assigned with scheduling constraints
• Participants randomly assigned to one of three roles
• Session 1: 5 40-min missions• Session 2: 3 40-min missions
10 Teams 10 Teams
9 Teams 10 Teams
3-5 weeks 10-13 weeks
Sam
eM
ixed
Com
posi
tion
Retention Interval
Mixed Condition
Session 1 Session 2
Retention
Interval
AVO PLO DEMPC AVO PLO DEMPC
Same Condition
Session 1 Session 2
Retention
Interval
AVO PLO DEMPC AVO PLO DEMPC
Team Retention and Composition
3-5
OR
10-1
3 W
eeks
All but Short-Intact teams suffer performance loss after the break
But a different story for Team Process (quality of team interactions)…
Team Process improves for mixed, but not intact teams after the break.
(There were no changes in knowledge after the break)
3-5
OR
10-1
3 W
eeks
Team Training MattersCross training (aligned with shared cognition) vs.
procedural/rigid training vs. Perturbation training (focused interactions)
Shared Mental Models
Assumptions• Individual is the unit of analysis• Measure individuals and aggregate• Increasing similarity or convergence over time is
associated with better teamwork • Focus on knowledge, static cognition (team mental
model, shared mental model)• A collection of knowledge experts should be an expert
team
+ +
Team Cognition =The collective knowledge of team members
Interactive Team Cognition
Team interactions often in the form of explicit communications are the
foundation of team cognition
ASSUMPTIONS
1) Team cognition is an activity; not a property or product
2) Team cognition is inextricably tied to context
3) Team cognition is best measured and studied when the team is the unit of analysis
US 2004 Olympic Basketball Team
"We still have a couple of days, but I don't know where we are," replied USA head coach Larry Brown … I've got a pretty good understanding of who needs to play. Now the job is to get an understanding of how we have to play."
A team of experts does NOT make an expert
team
Collaborative skill is not additive
US 1980 Olympic Ice Hockey Team
Herb Brooks and 20 young “no-names” won the 1980 Olympic Gold Medal in Ice Hockey
An expert team made up of no-names…
A Multi-Level Systems Perspective
• Micro -individual• Meso – team, group• Macro - organization, population
Borner, Contractor, Falk-Krzesinski, et al., 2010
Micro Level: Challenges
• Who should engage in team science?– Risks of early career tenure-track scientists
• Who should be on the team?– Team composition– Team assembly
• Faultlines and subgroups
Micro Level: Support
• Recommender systems • Research networking systems• Matching task to team assembly
Meso Level: Challenges
IPO Model (Hackman, 1987)
Four Phase Model of Transdisciplinary Research (Hall, Vogel Stipelman, 2012)
Meso Level: Challenges
Team Process Behaviors•Communication – shared mental models•Coordination•Conflict Resolution•Back-up Behavior•Situation Assessment
Meso Level: Support
•Training•Leadership•Technology•Tools for Team Science
– NCI Team Science Toolkit
Macro Level: Challenges
• Organizational rewards for team science• Disciplinary culture• Geographic dispersion• Complexity of multi-team systems• Mis-aligned goals
Team Charters• Communication plan between teams
(modes, media, who to whom)• Plan for regular interactions• Plan for leadership – shared• Identify boundary spanning individuals
Asencio, Carter, DeChurch, et al., 2012