research methods for the learning sciences c term, 2010 january 15, 2010

Research Methods for the Learning Sciences

C term, 2010January 15, 2010

Today’s Class

• Administrative Stuff• Philosophical Underpinnings I• Probing Question for Wed, Jan. 20• Survey

Administrative Stuff

• Did everyone sign up for an ISP (unless you are auditing)?

• If not, and you want to receive credit, please talk to me after class

Syllabus

• Everyone has downloaded this off the web, right?

Required Texts

• None

Recommended Texts Available Used on Amazon for <$40 total

• Rosenthal, R., Rosnow, R.L. (1991) Essentials of Behavioral Research: Methods and Data Analysis: 2nd edition. Boston, MA: McGraw-Hill.

• Simon, H.A. (1999)Sciences of the Artificial: 3rd edition. Cambridge, MA: MIT Press.

• Ferguson, G.A.(1971)Statistical Analysis in Psychology and Education. New York, NY: McGraw-Hill.

Recommended Readings[available online, check your email]

• Hamming, R. (1986) You and Your Research. Presentation at Bell Labs.

• Agre, P. (2001) Networking on the Network. Manuscript online.

• Bem, D.J. (2002) Writing the Empirical Journal Article. The Compleat Academic: A Career Guide. Washington, DC: American Psychological Association.

Required Readings

• Listed in the course schedule on the web

• Wed, Jan. 20 readings now online

Required Readings

• Are not actually all that required

Required Readings

• This is a Research Methods class• I have intentionally assigned more reading than

you can realistically do 100.0% of

• I expect you to be strategic and apply meta-cognition

• To decide what is absolutely crucial• And what you should skim to be prepared for

class discussion and for when you need the method in 3 years

Course Goals

• Learn data collection, design, and analysis methodologies that are particularly useful for scientific research in education

• Learn how to apply these methods to your own research program

• Be able to accurately select a valid method for answering a specific research question, and to defend that choice effectively but collegially

• Draw conclusions at the appropriate level of generality from the results obtained from a method

• Be able to identify the contributions and significance of research from other research paradigms

Assignments

• 5 Assignments• You will apply methods we’ve learned in class

to real data and/or research questions– If you want, you can use your own data and

research questions!– Or you can use mine

• Each assignment counts for 15% of grade• Late policy and turn-in policy is in your

syllabus

Assignments

1. QUANTITATIVE FIELD OBSERVATION/TEXT REPLAYS

2. INTERVIEW/CONTEXTUAL INQUIRY3. THINK ALOUD4. COGNITIVE MODELING5. TRANSFER/PREPARATION FOR FUTURE

LEARNING

Probing Questions

• Will be given out at the end of every class (or almost every class)

• Write a 1-3 paragraph response (or follow the directions)

• 15% of grade total• Late policy and turn-in policy is in your

syllabus

Class Participation

• 10% of final grade

Examinations

• There will be an incredibly painful and grueling examination on April 1

• Fortunately, it will count for 0% of your grade

Plagiarism and Cheating

• Don’t do it

• If you have any questions about what it is, talk to me before you turn in an assignment that involves either of these

• University regulations will be followed to the letter

Accommodations for Students with Disabilities

• See syllabus and then see me

Surveys

• Periodically, I will put up a survey on the web so that you can anonymously give me feedback on the previous class or assignment

• Please fill these out, as it will help me improve this course for you!

Questions

• Any questions on the syllabus, schedule, or administrative topics, before we launch into the fun?

Today’s Class


Anderson et al versus Greeno

John Anderson

• Leader of development of ACT-R Theory– Pre-eminent attempt to develop a unified theory

of cognition

My grandadvisor

Lynne Reder

• Extremely influential researcher in memory and cognition

Herbert Simon

• Nobel Laureate in Economics• One of the “fathers” of both Cognitive Science

and Artificial Intelligence• Important Philosopher of Design and Applied

Science

Jim Greeno

• Key theoretician of situationalist perspective• Former editor of Cognitive Science

Fairly unpleasant rhetorical practices on both sides

• Use of straw-man arguments, name-calling, comparison to unpopular researchers

• We’ll come back to this at the end of the semester

• But for now…

What are the scientific differences between Greeno and Anderson et al?

What are the scientific differences between Greeno and Anderson et al?

• Understanding learning in context as part of a system, versus understanding learning as something that an individual does

• Arguing for learning in context versus abstract transferrable learning

• Arguing for the decomposability of learning phenomena versus arguing for a holistic/systems perspective

Questions (according to Greeno)Cognitive Situative

How tightly bound is the knowledge to the context in which it is acquired?

Does activity that occurs in one type of situation have aspects that were learned as practices and interactions with the resources available in that type of situation, and does it have aspects that were learned as practices and interactions with resources in some quite different type of situation?

Will complex skills be acquired more successfully if instruction in various independent subskills is presented separately or in situations where all of the subskills are needed? In particular, will skills of complex social activities be learned more successfully if their independent subskills are learned in situations involving individual practice?

Which combinations and sequences of learning will prepare students best for the kinds of participation in social practices that we value most and contribute most productively to the development of students' identities as learners?

Does knowledge transfer between tasks? When someone has become more successful at participating in one kind of situation, are there other kinds of situations in which that person would be more adept?

What are the relative advantages and disadvantages of abstract instruction, as op-posed to instruction for specific activity, especially for jobs?

What kinds of abstract representations can contribute productively to meaningful, general learning?

Which group of authors is more comprehensible?

• Anderson et al• Greeno• Both are comprehensible• Neither are comprehensible

What are the differences in writing style between the two groups of authors?

What are the differences in writing style between the two groups of authors?

• Decomposed linear argument versus dialectic

A key difference between Greeno and Anderson et al

• Should learning be understood• in context as part of a system– can not be decomposed or studied in parts

• or as something that an individual does– decomposable, meaningfully can be studied in

parts

This is not just a debate between Greeno and Anderson et al

• And it is not just a debate from 13 years ago

• It is a very ongoing and current debate, not just in education but elsewhere

Buchanan and McKeon’s Framework

• “Ways to design”• “The cross of pane”• Neo-Aristotelianism

Dick Buchanan

• Founder of journal Design Issues• Inventor of term “wicked problem”• Key modern theoretician of design

Richard McKeon

• Key 20th century philosopher • Referred to as the “Evil Professor” by Robert

Pirsig in Zen and the Art of Motorcycle Maintanance

ENTITATIVE

HOLISTIC

ESSENTIALISTEXISTENTIALIST


ENTITATIVE

HOLISTIC



In education, called “the Cognitive Approach” by its friends

ENTITATIVE

HOLISTIC



You can understand things “in themselves”

ENTITATIVE

HOLISTIC



You can understand things separately from their contexts

ENTITATIVE

HOLISTIC



Individuals learn skills and concepts

ENTITATIVE

HOLISTIC



Key method: REDUCTIONISM

What is reductionism?

ENTITATIVE

HOLISTIC



We can understand complex phenomena by understanding their parts and how those parts interact

ENTITATIVE

HOLISTIC



Emergent phenomena can be understood by first understanding the motivating factors and processes

ENTITATIVE

HOLISTIC



Bloom

ENTITATIVE

HOLISTIC



ENTITATIVE

HOLISTIC



Called “the ontological approach” by Buchanan, and Gestalt theory by others

ENTITATIVE

HOLISTIC



In education, called “Situationalism” by its friends

ENTITATIVE

HOLISTIC



Things can only be understood by understanding the systems they operate in

ENTITATIVE

HOLISTIC



Learners are enculturated into practices that communities agree are correct

ENTITATIVE

HOLISTIC



Complex phenomena must be understood as wholes; they can not beunderstood in terms of their component parts

ENTITATIVE

HOLISTIC



Emergent phenomena must be understood in themselves

ENTITATIVE

HOLISTIC



Key method: DIALECTIC

What is dialectic?

ENTITATIVE

HOLISTIC



Truth is found through the struggle of opposing hypotheses and their eventual unification in a richer synthesis (Hegelian-Kantian dialectic)

ENTITATIVE

HOLISTIC



Truth is found through positing overly abstract hypotheses, finding evidence that negates them, and coming to a newer concrete theory (Hegelian dialectic)

ENTITATIVE

HOLISTIC



Change is governed by historical properties (historicism) where contradictions and states repeat themselves but in continual refinement (material dialectic)

ENTITATIVE

HOLISTIC



Vygotsky Barab Pirsig

Another key dimension

• Is there a reality out there that we want kids to learn about?

• Are we just teaching cultural practices, or are we teaching real things?

ENTITATIVE

HOLISTIC



ENTITATIVE

HOLISTIC



Doesn’t really have a name that its friends call it, in education

ENTITATIVE

HOLISTIC



Maybe I would call it “The Curricular Approach”

ENTITATIVE

HOLISTIC



Its enemies call it “Instructionism” (cf. Papert, 1981)

ENTITATIVE

HOLISTIC



There is a real fundamental reality to the universe that students can learn about

ENTITATIVE

HOLISTIC



Or, in the “Pragmatic Essentialism” form, there are beliefs, skills, and practices that are so core to a specific culture and created-world as to be real for all practical purposes

ENTITATIVE

HOLISTIC



Essentialism is implicit in the focus on mathematical skills, scientific skills, and literacy

ENTITATIVE

HOLISTIC



Knowledge can be measured and compared, because there’s some real reference that two students can be compared according to

ENTITATIVE

HOLISTIC



If you use the MCAS, you’re an essentialist in practice

ENTITATIVE

HOLISTIC



If you think the MCAS is crap because it doesn’t measure real math/science skill/knowledge, you’re still an essentialist

ENTITATIVE

HOLISTIC



Dewey

Gobert

Gagne

ENTITATIVE

HOLISTIC



ENTITATIVE

HOLISTIC



In education, called “Constructionism” by its friends

ENTITATIVE

HOLISTIC



“Constructionism” is not the same thing as “Constructivism”

ENTITATIVE

HOLISTIC


Buchanan and McKeon’s Framework“Constructivism” argues that students learn actively rather than passively, and learn best by doing. It’s pretty much orthogonal to the dimensions shown below.

ENTITATIVE

HOLISTIC



Everyone defines their own reality in their own individual way

ENTITATIVE

HOLISTIC



Agreed-upon conventions are simply agreed-upon conventions and reflect no fundamental reality

ENTITATIVE

HOLISTIC



0, 1, and 47 are social constructions; blue and red are social constructions

ENTITATIVE

HOLISTIC



And they’re actually not even the same thing in our heads

ENTITATIVE

HOLISTIC



So when you and I think of “blue” we’re thinking of different things

ENTITATIVE

HOLISTIC



Because of this, measuring learning is fundamentally misguided – there is no real commonality to compare – even if we manifest the same behavior, the cognitive reality is unreduceable

ENTITATIVE

HOLISTIC



If you think that the MCAS is junk, and never can be improved in any way, because tests don’t measure anything that matters, you’re an existentialist in practice

ENTITATIVE

HOLISTIC



School should not focus on math or literacy skills, it should focus on “learning how to learn” and generating passion for learning, because everyone will be passionate and skilled at different things

ENTITATIVE

HOLISTIC



So if you want to measure something, measure passion or measure the great things a student can do and the surprising connections they make

ENTITATIVE

HOLISTIC



Papert

Edelson

Crossing the Center

• Many researchers are at the diagonals of this graph rather than the ends

• Some researchers do not fit neatly in these categories– e.g. they use both holistic and entitative methods when they

seem useful

• That said… Buchanan claims that most researchers and designers have a strong preference, and I have generally found that to be true– I *definitely* have preferences towards a diagonal– And the only education theorist I’ve ever seen who managed to

write in both holistic and entitative ways effectively was Vygotsky

Methods

• Most methods are tightly identified with one (or at most two, at a diagonal) of these philosophical positions

• We’ll re-visit this issue during the semester as we study specific methods

Why does this matter?

• It’s important to know what kinds of methods can answer what kinds of questions

• And it’s important to know where other researchers are coming from

At the extreme

• Researchers who are strongly identified with one position tend to find research drawing on the opposite set of assumptions– incomprehensible, outdated, acting in bad faith,

or even evil

• We’ll study examples of that at the end of the semester, but one classic one is Papert’s late-1980s claim that “Instructionism” is like the Soviet Union and “constructionism” is like glasnost and perestroika for the American educational system

At the extremes

• A surprising number of vicious attacks in educational theory can be linked to one of the two divisions discussed today

At the end of the semester

• We will discuss why it’s important to know your own assumptions

• Why it’s important to be able to argue for them

• And why it’s important to be able to learn from research conducted with assumptions different than yours

Today’s Class


Probing Question

Where do you fit on this diagram?

ENTITATIVE

HOLISTIC


No explanation is necessary, just email me a modified version of this slide showing where you fit

ENTITATIVE

HOLISTIC


Today’s Class


Please complete this surveybefore the next class

• http://www.surveymonkey.com/s/YWRGT5M

The End

research methods for the learning sciences c term, 2010 january 15, 2010

Documents

class slide

survey slide

required texts

recommended texts available

learning sciences c

research methods