communicating scientific research effectively to a broad...

Communicating Scientific Research Effectively to a Broad AudienceChem 691 Graduate SeminarOctober 11, 2011

Hannah SevianAssociate ProfessorChemistry DepartmentUniversity of Massachusetts Boston

DisclaimerThis workshop is based on work supported by National Science Foundation (NSF) awards DGE-0231638 and DGE-0538445. The work continued to be supported by the NSF as part of the author’s independent research and development plan while employed by the NSF. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the NSF.

Problem• How can you tell when a presentation of scientific

research is good?• How can you tell when it is poor?

Workshop objectives• To be able to recognize aspects of science

presentations• To be able to assess the quality of various aspects of

a scientific presentation• To understand the general nature of what can be

improved in a scientific presentation• To pinpoint specific ways to improve explaining

science to a general audience

Agenda• Where did this rubric come from?• Norm rubric usage with TED videos• Some results from TED videos (submitted by graduate

students in Arizona earlier this year)• Findings from research study that produced the rubric• Try it: Use the rubric to hone a scientific explanation

• Form groups of 3 people• One person presents an “elevator speech” about his/her

research project, and the other two use the rubric to offer feedback

• Debrief

A more complete view of the problem

• Efforts by scientists to communicate the nature and results of scientific research are hindered by lack of training and emphasis on scientific communication to a general audience

• Federal agencies have begun to place increased emphasis on scientists ensuring “broader impacts” of their research

• A significant first step toward having broader impacts is understanding how to explain science effectively to a general audience

The research questions• How do science graduate students improve in their

ability to explain their own research to a general audience of non-scientists?1. What are the components of an

effective scientific explanation?2. Can the quality of an individual

component be measured?3. How do the components

contribute to the effectivenessof the explanation?

Method of the study• We asked science graduate students impromptu to,

“explain your own research to an audience of non-scientists in 3 to 5 minutes,” and we videotaped the presentations

• Explanations prior to involvement in an NSF GK12 program and again after one year

• Participants included 32 graduate students over 4 years, representing diverse cultural backgrounds and various stages of M.S. and Ph.D. programs in science departments

Grounded theory for model development• Theory developed inductively from body of data

• Process of comparative orientation

• Resulting theory at least fits one data set perfectly (contrast: first principles theories)

• Quality of a theory evaluated by process by which it is constructed (contrast: ability to explain new data)

The rubricRuns parallel to what we know about effective science teaching –that it requires three kinds of knowledge:

1. Science content knowledge

2. Pedagogical knowledge

3. Integration of content and pedagogy in the service of a clear explanation

1. Science content knowledgeFactual knowledge and processes and how well that knowledge is understood in broader contexts. Assesses accuracy and depth, including how well the scientist portrays the overall organization of knowledge.

• Factual knowledge

• Evidence of organization of knowledge by the guiding principles of the discipline

• Ability to transfer knowledge to broader contexts

2. Pedagogical knowledgeThe knowledge and skill involved in explaining major concepts involved in the scientist’s research. Assesses methods scientists employ to communicate their knowledge orally to an audience, with written and/or other media to support in real-time.

• Structure and balance of presentation

• Response to the audience

• Choice of language

• Technical skill of presentation and use of media

3. Integration of content and pedagogyAssesses the ability to integrate content and pedagogy in the service of a clear, coherent, and engaging explanation of scientific research

• Development of appropriate mental images to support explanation

• Tactical use of media

• Scaffolded explanation

Two TED videos of graduate studentsPike: Climate change Behncke: Bonobo apes

Holistic assessment using “Integration” part of rubric - Look for:• Tactical use of media• Development of appropriate mental images to support explanation• Scaffolded explanation• Also remember to look for gestures because they give clues about

what a person means

Pike: Climate Change• Integration of content

and pedagogyAssesses the ability to integrate content and pedagogy in the service of a clear, coherent, and engaging explanation of scientific research 0%

10%

20%

30%

40%

50%

60%

70%

1 Weak level ofcontent knowledge

2 Poorly connectedcontent knowledge

3 Sufficient level ofcontent knowledge

4 Sophisticatedlevel of contentknowledge

Tactical Use of Media

0%

10%

20%

30%

40%

50%

60%

70%





Use of Mental Images

0%

10%

20%

30%

40%

50%

60%

70%





Scaffolding

Behncke: Bonobo Apes• Integration of content

and pedagogyAssesses the ability to integrate content and pedagogy in the service of a clear, coherent, and engaging explanation of scientific research 0%

10%

20%

30%

40%

50%

60%

70%





Tactical Use of Media

0%

10%

20%

30%

40%

50%

60%

70%





Use of Mental Images

0%

10%

20%

30%

40%

50%

60%

70%





Scaffolding

Comparison of Pike and Behncke• Integration of content

and pedagogyTactical use of media: Knowing when to use particular media

Behncke

Pike0%

10%

20%

30%

40%

50%

60%

70%

1 Nointegration ofcontent andpedagogy

2 Weakintegration ofcontent andpedagogy

3 Developingability tointegrate

content andpedagogy

4 Competent atintegration ofcontent andpedagogy

Integration: Tactical Use of Media Behncke

Pike

AveragesBehncke 3.71Pike 3.47

(*)


and pedagogyUse of mental images: Choice of appropriate imagesthat give theaudience resourcesfor thinking aboutscientific ideas

Behncke

Pike0%

10%

20%

30%

40%

50%

60%

70%




content andpedagogy


Integration: Use of Mental ImagesBehncke

Pike


(*)


and pedagogyScaffolding: Weaving examplestogether to buildan explanation

Behncke

Pike0%

10%

20%

30%

40%

50%

60%

70%




content andpedagogy


Integration: Scaffolding

Behncke

Pike


Science content knowledgeFactual knowledge and processes and how well that knowledge is understood in broader contexts. Assesses accuracy and depth, including how well the scientist portrays the overall organization of knowledge.

• Factual knowledgeFactual knowledge and processes and how well that knowledge is understood in broader contexts. Assesses accuracy and depth, including how well the scientist portrays the overall organization of knowledge

• Organization of knowledge by the guiding principles of the disciplineAbility to break down concepts into coherent chunks reflects an underlying organization of knowledge and understanding of connections between discrete categories of knowledge

• Ability to transfer knowledge to broader contextsUse of the knowledge in and relation to many contexts

Pike: Climate Change• Content knowledgeFactual knowledge and processes and how well that knowledge is understood in broader contexts. Assesses accuracy and depth, including how well the scientist portrays the overall organization of knowledge.

0%

10%

20%

30%

40%

50%

60%

70%





Factual Knowledge

0%

10%

20%

30%

40%

50%

60%

70%





How Knowledge isUnderstood

0%

10%

20%

30%

40%

50%

60%

70%





Ability to TransferKnowledge

Behncke: Bonobo Apes• Content knowledgeFactual knowledge and processes and how well that knowledge is understood in broader contexts. Assesses accuracy and depth, including how well the scientist portrays the overall organization of knowledge.

0%

10%

20%

30%

40%

50%

60%

70%





Factual Knowledge

0%

10%

20%

30%

40%

50%

60%

70%





How Knowledge isUnderstood

0%

10%

20%

30%

40%

50%

60%

70%





Ability to TransferKnowledge

Comparison of Pike and Behncke• Content knowledgeFactual knowledge: Facility with manyfacts aboutthe science


Behncke

Pike0%

10%

20%

30%

40%

50%

60%

70%

1 Weak level ofcontent

knowledge

2 Poorlyconnectedcontent

knowledge

3 Sufficientlevel of contentknowledge


Content Knowledge: Factual Knowledge Behncke

Pike

*

Comparison of Pike and Behncke• Content knowledgeHow knowledge isunderstood: Ability to break down concepts into coherent chunks reflects an underlying organization of knowledge andunderstanding ofconnectionsbetween discretecategories of knowledge


Behncke

Pike0%

10%

20%

30%

40%

50%

60%

70%


knowledge


knowledge



Content Knowledge: How Knowledge is Understood

Behncke

Pike

Comparison of Pike and Behncke• Content knowledgeAbility to transferknowledge: Use of theknowledge inand relation tomany contexts


Behncke

Pike0%

10%

20%

30%

40%

50%

60%

70%


knowledge


knowledge



Content Knowledge: Ability to Transfer Knowledge

Behncke

Pike

*

Pedagogical knowledgeThe knowledge and skill involved in explaining major concepts involved in the scientist’s research. Assesses methods scientists employ to communicate their knowledge orally to an audience, with written and/or other media to support in real-time.

• Structure and balanceStructure is the logic and flow of an explanation, while balance describes the time spent on each part of the explanation

• Response to audienceCustomizing the explanation to the audience, taking account of prior knowledge, providing cues for engagement

• Clear choice of languageLanguage at appropriate level, unambiguous and concise, words that audience unfamiliar with are defined clearly

• Skill of presentation: Technical use of mediaUse of uncluttered media to emphasize main points and assist audience in understanding main points

Pike: Climate Change• Pedagogical skillKnowledge and skill involved in explaining major concepts

0%

10%

20%

30%

40%

50%

60%

70%

1 Application ofpedagogicalelements notapparent

2 Application ofpedagogicalelements isemerging butinadequate

3 Application ofappropriatepedagogicalelements ispresent butinconsistent

4 Application ofappropriatepedagogicalelements isconsistent

Structure & Balance

0%

10%

20%

30%

40%

50%

60%

70%





Response to Audience

0%

10%

20%

30%

40%

50%

60%

70%





Clear Choice of Language

0%

10%

20%

30%

40%

50%

60%

70%





Skill of Presentation

Behncke: Bonobo Apes• Pedagogical skillKnowledge and skill involved in explaining major concepts

0%

10%

20%

30%

40%

50%

60%

70%





Structure & Balance

0%

10%

20%

30%

40%

50%

60%

70%





Response to Audience

0%

10%

20%

30%

40%

50%

60%

70%





Clear Choice of Language

0%

10%

20%

30%

40%

50%

60%

70%





Skill of Presentation

Comparison of Pike and Behncke• Pedagogy Structure & balance: Structure is the logic and flow of an explanation, while balance describes the time spent on each part of the explanation


Behncke

Pike0%

10%

20%

30%

40%

50%

60%

70%





Pedagogy: Structure & Balance Behncke

Pike

Comparison of Pike and Behncke• Pedagogy Response to audience: Customizing the explanation to the audience, taking account of prior knowledge, providing cues for engagement


Behncke

Pike0%

10%

20%

30%

40%

50%

60%

70%





Pedagogy: Response to Audience Behncke

Pike

*

Comparison of Pike and Behncke• Pedagogy Clear choice of language: Language at appropriate level, unambiguous and concise, words that audience unfamiliar with are defined clearly


Behncke

Pike0%

10%

20%

30%

40%

50%

60%

70%





Pedagogy: Clear Choice of Language Behncke

Pike

(*)

Comparison of Pike and Behncke• Pedagogy Skill of presentation: Use of uncluttered media to emphasize main points and assist audience in understanding main points


Behncke

Pike0%

10%

20%

30%

40%

50%

60%

70%





Pedagogy: Skill of Presentation Behncke

Pike

Findings from the research study1. Effective explanation of science is developed in layers

o When an explainer is strong in only one type of knowledge (content or pedagogy), there is a clear transition in the person’s explanation

2. Only when both pedagogy and content are strong is a scientific explanation effective

o Science explanations in this category exhibit development of powerful mental images, tactical use of media to support explanation, and scaffolded development of concepts

3. Presentation skills add an extra layer that can cause a good presentation (transcript) to fail (video) and a poor presentation (transcript) to appear to succeed (video)

4. Gestures sometimes reveal deep content understanding that the explainer is unable to articulate verbally

Characteristics of four types of explanations in science

Weak content knowledge (of own research)

Strong content knowledge (of own research)

Weak pedagogical knowledge

Unclear from start to finish. Engaging introduction that relates to audience but is unrelated to research, or big picture about research that is not relevant to audience.

Clear transition to more complex scientific explanation inappropriate for audience.

Strong pedagogical knowledge

Strong introduction that establishes structure, and is engaging, clear, and relates the research to the audience.

Clear transition to explaining own science, unable to explain clearly.

Clear logical structure, story maintained throughout explanation.

Explanation scaffolded throughout. No transition.

REORGANIZEForm groups of three people• One person will give a maximum 3-minute

“elevator speech” about his/her research project

• The other two people should use the rubric as guidance to provide feedback:• What does the presenter do well?• What can the presenter focus on improving?

Review objectives: Did we succeed?• To be able to recognize

aspects of science presentations

• To be able to assess the quality of various aspects of a scientific presentation

• To understand the general nature of what can be improved in a scientific presentation

• To pinpoint specific ways to improve explaining science to a general audience

Conclusions andfuture thinking• The rubric can be used as a

tool to help you hone yourabilities in presenting aboutscience to a general audience

• Gestures can be useful or distracting, and they also give information about the presenter’s abilities

• Generally, when a presentation is not as good as it could be, it is because the presenter is either better at pedagogy or better at content knowledge, and it’s usually possible to tell which one it is

• Using the “Integrated content and pedagogy” part of the rubric is a quick way to assess a presentation• Remember: 1) create mental images (gestures can help), 2) use

media tactically, 3) scaffold your explanations logically with good size steps

Acknowledgments• Graduate students at UMass Boston who participated in this

study• Collaborator: Lisa Gonsalves (Curriculum & Instruction

Department, UMass Boston)• Inspiration and helpful discussions: Robert Chen (EEOS

Department, UMass Boston)• National Science Foundation

Thank you for participating today

communicating scientific research effectively to a broad...

Documents