practical strategies for student metacognition

Practical Strategies for Student Metacognition

Ed Nuhfer – Director of Faculty Development, Director of Educational

Assessment and Professor of Geology (University of Wisconsin, University of

Colorado, Idaho State University, California State University (retired)

enuhfer@earthlink.net Phone 208-241-5029

Technology, Innovations, & Pedagogy ConferenceCSU Fresno

August 18, 2014

“High level teaching" is "…teachers teaching the students something,

instructing them in how to produce something, and giving them instruction

as to the processes of learning.”

Hattie, J. (2009). Visible Learning: A Synthesis of over 800 Meta-Analyses Relating to Achievement. New York, New York: Routledge. P. 11.

Metacurriculum for Metacognition Instruments Knowledge or Skill

Learning-across-curriculum modules

Strategies and developing a learning philosophy

Reading Reflections Reflection & Monitoring

Exam WrappersReflection, Monitoring & Evaluation

Knowledge SurveysGoal-setting, Monitoring & Evaluation Developing self-assessment & self-efficacy

Learning JournalsEvaluation & Goal-setting Developing self-assessment & self-efficacy

Metadisciplinary Concept Inventories (Assessment)

Reasoning, respect for diverse ways of knowing, overarching philosophy for becoming educated

Available Learning-Across-the-Curriculum Modules From CSU Faculty Developers

Used by permission of S. E. Petersen. Positron emission tomographic studies of the processing of single words.

Tools and Scales of Learning Challenges

“Exam Wrappers”

1. What did I do well on and why?2. What did I do less well and why?3. When I encounter a similar challenge, what strategy will

I use to perform better?

95% for the exam plus 5% for submitting the following metacognitive reflection:

Reflective Learning Journals (1)

1. Summarize your assumptions, beliefs, understanding that you had when you first encountered the topic assigned here, before you started to engage it during learning it in this course

1. Assessing what one thinks at the start of a task is a common theme in metacognition. A free writing exercise given at the introduction of a new topic is ideal for helping students capture their baseline knowledge and archive that for later use in a journal entry.

Reflective Learning Journals (2)

2. Disclose new vocabulary that you acquired during your study of this topic. Include vocabulary that you may have acquired as part of your process in items 4-6 below.

2. Building vocabulary increases reading comprehension and personal ability to access information.

Reflective Learning Journals (3)3. Discover and describe

connections that you can make between the topic and several other class sessions, exercises, readings, homework. Note in SPECIFICS what you found from making these connections that informed or expanded your understanding of the topic. Be sure to note how this learning is relevant to one or more of the published learning outcomes for the course. Refer to the specific outcome(s).

3. Metacognitive reflection uncovers connections that are otherwise difficult to access. Practice in seeking connections between lessons and larger ideas is immensely valuable.

Reflective Learning Journals (4)

4. So as avoid depending on materials given to you in class, take action on your own behalf by looking to other resources. To provide evidence of this learning, describe IN SPECIFICS what you learned and from what source(s).

4. Assigning students to go beyond what authority (instructors) offers them moves beginning students to higher-level stages of thinking. Metacognition trains independent thinking.

Reflective Learning Journals (5)

5. “Close the loop” by comparing what you knew in #1 against what you know now.

5. Seeing one's growth affirmed builds self-efficacy.

Journal template furnished as “blueline helpsheet”

Journal template furnished as “blueline helpsheet”

• Download it from http://profcamp.tripod.com/rljhelpsheet.rtf.

Knowledge surveys

• Metacognitive self-assessments – If I think I can do this, how well can I do this?– Can I look at the items and start to distinguish

which emphasize knowledge, skills or reasoning?• For construction, consult the tutorials at http://elixr.merlot.org/assessment-evaluation/knowledge-surveys?noCache=191:1306864598

Knowledge surveys…

Our challenge is to keep students in constant

contact with their metacognition…• KS ITEM – COURSE CONTENT: 0 1 2 I can explain why it was

necessary to first understand the principles of original horizontality, superposition, and crosscutting relationships before scientists could begin to derive the theory of plate tectonics.

• KS ITEM — METADISCPLINARY OUTCOME: 0 1 2 I can employ examples to illustrate key differences between the ways of knowing of science and of technology.

• KS ITEM — STAGE IN DEVELOPMENT OF THINKING: 0 1 2 My friend tells me: "Because everyone has a right to his or her opinion, we respect others by accepting all opinions as equally valid.” I can recognize the stage of intellectual development revealed by that statement.

Hattie (2009, p 39 - 43) found present understanding of level of achievement as the most significant attribute of students for meeting their future goals.

Self-Efficacy

• “If I have the belief that I can do it, I shall surely acquire the capacity to do it, even if I may not have it at the beginning.” Mahatma Gandhi

KSs as an aid to teaching self-efficacy• Self-efficacy—realistic belief in self regarding the

proficiency that one can attain with instruction and support.

• Alfred Bandura established strong self-efficacy as one of the most important predictors of student success (www.des.emory.edu/mfp/self-efficacy.htm)

• PRE-COURSE: “Good heavens! Will we be able to learn ALL this?"

• POST COURSE: "I really did learn that much.”

Correlations between performance and several metacognitive self-assessments

(n = 1011)First

Impression Global Self-assessment

Knowledge Survey item-by-item Self-Assessment

Post-Knowledge

Survey Overall Self- Assessment

Post-SLCI Overall Self-assessment

(*n=590)

SLCI Actual Score

r = 0.29 r = 0.58 r = 0.45 r = 0.62*

Average 72 %

Average 75%

Average 79%

Average 76%

Average* 76%

Metadisciplinary Ways of Knowing

• You have a handout with outcomes for six metadisciplines

• If not, grab it from http://profcamp.tripod.com/metadoutcomeslist.pdf

Example of the reasoning component of “Science literacy”

• Insert your own metadiscipline’s outcomes in place of “science” as you consider what follows.

• If a citizen understands science’s way of knowing, how can we recognize this? How can students recognize when they are “getting it?”

Concepts for Citizen Literacy in the Metadiscipline of Science1. Science explains physical phenomena based upon testable information about

the physical world.2. In modern life, science literacy is important to both personal and collective

decisions that involve science content and reasoning.3. Doubt plays necessary roles in advancing science.4. Scientists use evidence-based reasoning to select which among several

competing working hypotheses best explains a physical phenomenon.5. A theory in science is a unifying explanation for observations that result from

testing several hypotheses.6. Peer review generally leads to better understanding of physical phenomena than

can the unquestioned conclusions of involved investigators.7. Science can test certain kinds of hypotheses through controlled experiments.8. All science rests on fundamental assumptions about the physical world.9. Science differs from technology.10. Scientific knowledge is discovered, and some discoveries require an important

history.11. Science employs modeling as a method for understanding the physical world.12. Scientific knowledge imparts power that must be used ethically.

Concepts restated as 12 Outcomes for Science LiteracyStudents will be able to…

1. Define the domain of science and determine whether a statement constitutes a hypothesis that can be resolved within that domain.

2. Describe through example how science literacy is important in everyday life to an educated person.

3. Explain why the attribute of doubt has value in science.

4. Explain how scientists select which among several competing working hypotheses best explains a physical phenomenon.

5. Explain how "theory" as used and understood in science differs from "theory "as commonly used and understood by the general public.

6. Explain why peer review generally improves our quality of knowing within science.

7. Explain how science employs the method of reproducible experiments to understand and explain the physical world.

8. Articulate how science’s way of knowing rests on some assumptions.

9. Distinguish between science and technology by examples of how these are different frameworks of reasoning.

10. Cite a single major theory from one of the science disciplines and explain its historical development.

11. Explain and provide an example of how modeling is used in science.

12. Explain why ethical decision-making becomes increasingly important to a society as it becomes increasingly advanced in science.

OK…Can we assess this stuff?

The Instrument • For each outcome

– Construct several concept inventory items.– Use established methods for drafting items that have been

developed in other concept inventories.

• In addition…– Test reasoning, not factual knowledge.– Administer the inventory under the conditions in which a

citizen will use common information.

…initial instrument constructed 2008-2010 through the collegial efforts of

• Edward Nuhfer, Faculty Development & Geology, Channel Islands

• Jerry Clifford, Physics, Channel Islands

• Christopher Cogan, Environmental Sciences & Resource Management, Channel Islands

• Anya Goodman, Biochemistry, San Luis Obispo

• Carl Kloock, Biology, Bakersfield

• Beth Stoeckly, Physics, Channel Islands

• Christopher Wheeler, Geology, Channel Islands

• Gregory Wood, Physics, Channel Islands

• Natalie Zayas, Science Education & Environmental Sciences, Monterey Bay

Science Literacy Concept Inventory

• Incorporates 25 validated items that map to the twelve concepts

• Reliability of .85• Tested on over 16,000 students in about 30

institutions

Outcome. Student can define the domain of science and determine whether a statement constitutes a hypothesis that can be resolved within that domain.

• Science explains physical phenomena based upon testable information about the physical world.

• Science is on a mission to refute religion; scientists study the paranormal; untestable statements are like scientific hypotheses.

Concept Some Misconceptions

Which of the following statements presents a hypothesis that science can now easily resolve?

A. Warts can be cured by holding quartz crystals on them daily for a week.

B. A classmate sitting in the room can see the auras of other students.

C. Radio City Music Hall in New York is haunted by several spirits.

D. People with chronic illnesses have them as punishment for past misdeeds.

Which of the following statements presents a hypothesis that science can now easily resolve?

A. Warts can be cured by holding quartz crystals on them daily for a week.

B. A classmate sitting in the room can see the auras of other students.

C. Radio City Music Hall in New York is haunted by several spirits.

D. People with chronic illnesses have them as punishment for past misdeeds.

What did we learn that made this so interesting?

• Starting with…Do experts (professors) outscore novices (students)?

YES! They do! Also, students on average do come to us with some science literacy: zero literacy = about 25% (random guessing).

Does knowing “stuff” advantage a person?

Professors in Metadisciplines by Mean Score %

We can learn about our students’ science literacy

Our current GE science courses don’t produce science literacy (N = 16,623)…

Development seems more apparent long term through ranks (N = 16,599)…

The major gains are breaks that may result from attrition as well as growth in ability to reason.

We can learn much about our students

First Generation Students: 65% vs 70% (N = 16,203)

Students from families privileged with higher education generally achieve higher.

English as a First Language: 62% vs 70% (N = 16,182)

Students whose first language is English generally achieve significantly higher scores.

Commitment to Interest in Science: 66% vs 72% (N = 16,190)

Differences are already significant in freshman year, and becomes increasingly so with each rank.

Gender-equality (n = 16,617)

And this holds true so far in our study within every ethnic group except one.

Middle Eastern (n = 373)

Ethnicities Show Some Profound Differences

They Differ Across Three Factors

Regression Equation to Account for Socio-economic advantages/disadvantages based on 16,263 undergraduates

• Corrected Score = 0.62424 - 0.032575 (First Generation) +0.06206 (English as Native Language) + 0.05323 (Science Interest).

Ethnicity adjusted for Socio-economic advantages

SLCI by Institutions with Other Measures

As Eric Gaze and the Quantitative Literacy test researchers have noted, these little literacy tests may be powerful predictors of student success.

Institutions

Given that our GE science courses don’t produce much increase in reasoning of science literacy…

For continued learning about metacognition, we recommend that participants enroll with the Improve with Metacognition forum at http://www.improvewithmetacognition.com/2014/07/