1 an application of the action research model for assessment preliminary report jsm, san francisco...
TRANSCRIPT
1
An Application of the Action An Application of the Action Research Model for Assessment Research Model for Assessment
Preliminary ReportPreliminary Report
JSM, San Francisco
August 5, 2003
Tracy Goodson-Espy, University of AL, Huntsville
M. Leigh Lunsford, University of AL, Huntsville
Ginger Holmes Rowell, Middle TN State University
2
OverviewOverview
• Action Research Model
• In the Context of a Collaborative Project
• Showing Results from a Specific Example
3
Action Research Model*Action Research Model*
• What is the problem? I.e., what is not working in the classroom?
• What technique can be used to address the learning problem?
• What type of evidence can be gathered to show whether the implementation is effective?
• What should be done next, based on what was learned?
*1999 - R. delMas, J. Garfield, B. Chance
4
Teaching Experiment CycleTeaching Experiment Cycle
Teaching Hypotheses; Curricular & Instructional
Choices
Class Implementation
& Feedback
Instructors’ Reflections and Curricular Modifications
5
Action Research ModelAction Research Model
• What is the problem? I.e., what is not working in the classroom?
• What technique can be used to address the learning problem?
• What type of evidence can be gathered to show whether the implementation is effective?
• What should be done next, based on what was learned?
6
What is the Problem? What is the Problem?
• Math majors who take only one course in “Probability and Statistics” are exposed to very little statistics
• Student understanding of complicated concepts– CLT, CI’s, Combinatorics, Baye’s Theorem, ...
• Increase reasoning and thinking instead of memorization
• Better preparation for careers
7
Action Research ModelAction Research Model
• What is the problem? I.e., what is not working in the classroom?
• What technique can be used to address the learning problem?
• What type of evidence can be gathered to show whether the implementation is effective?
• What should be done next, based on what was learned?
8
What Technique can be used to What Technique can be used to Address the Learning Problem?Address the Learning Problem?
NSF DUE A/I Collaborative Research Award– Adaptation & Implementation of Activity & Web-
Based Materials in Post-Calculus Introductory Probability & Statistics Courses
– PI’s• Tracy Goodson-Espy, University of AL, Huntsville• M. Leigh Lunsford, University of AL, Huntsville• Ginger Holmes Rowell, Middle Tennessee State University
Big Picture
*This project is partially support by the National Science Foundation. The project started in June 2002 and continues through August 2004..
9
A Collaborative ApproachA Collaborative Approach
Athens State Univ.M. Leigh Lunsford
Middle Tenn. St. Univ.Ginger Holmes Rowell
Univ. of Alabama, HuntsvilleTracy Goodson-Espy
A&I Materials into Post Calculus Prob/Stat Courses
Provide Objective Independent Assessment of A&I
10
The Materials for A&IThe Materials for A&I
• ““A Data-Oriented, Active Learning, Post-A Data-Oriented, Active Learning, Post-Calculus Introduction to Statistical Calculus Introduction to Statistical Concepts Methods, and Theory (SCMT)”Concepts Methods, and Theory (SCMT)”
• A. Rossman, B. Chance, K. Ballman• NSF DUE-9950476
• ““Virtual Laboratories in Probability and Virtual Laboratories in Probability and Statistics (VLPS)”Statistics (VLPS)”
• K. Siegrist• NSF DUE-9652870
11
What Technique can be used to What Technique can be used to Address the Learning Problem?Address the Learning Problem?
• Increase statistical content– Change course description & course number
• Understand complicated concepts and increase “thinking” instead of memorization – Change course materials used
• Better preparation for careers– Integrate technology, group work, report writing
Specific Techniques
12
Action Research ModelAction Research Model
• What is the problem? I.e., what is not working in the classroom?
• What technique can be used to address the learning problem?
• What type of evidence can be gathered to show whether the implementation is effective?
• What should be done next, based on what was learned?
13
What Evidence can be Gathered to What Evidence can be Gathered to Show Implementation is Effective?Show Implementation is Effective?
• Student’s self perception of learning of concept (survey results)
• Teacher perception of student learning– In-class student feedback during the activity
(continuous monitoring)– Follow-up in-class quiz– Student reports – Test/Exam questions
• Student attitude survey results
14
Evidence Continued:Evidence Continued:One ExampleOne Example – Central Limit Theorem – Central Limit Theorem
“Sampling Distributions of Sample Means” Computer Laboratory Simulation Activity– A. Rossman, B. Chance, K. Ballman
• ““A Data-Oriented, Active Learning, Post-Calculus A Data-Oriented, Active Learning, Post-Calculus Introduction to Statistical Concepts Methods, and Introduction to Statistical Concepts Methods, and Theory (SCMT)”Theory (SCMT)”
– Spring 2003: used 3 of 4 examples from this activity, students wrote reports, question on test
15
Evidence (continued) Evidence (continued) CLT ExampleCLT Example
• Students’ self perception of learning– 80% (n=25) of students remembered it as an
“activity that aided learning” (Fall 02 – instructor lecture & 1 SCMT example: 23%)
– “My understanding of the Central Limit Theorem”• Survey question of their self reported knowledge on a
scale of 1 (low knowledge) to 5 (high knowledge)• Mean response = 4.6, stdev = .64, n = 27
(Fall 02: mean = 3.8, stdev =.90, n = 25)
16
Evidence (continued) Evidence (continued) CLT ExampleCLT Example
• Teacher’s perception of student learning• In-class monitoring
– Overall minimal difficulty with computer lab– Needed feedback on their understanding of the
concept • Gave in-class (no credit) unannounced quiz at
beginning of following class• Not enough students had gotten far enough on the
activity to provide conclusive evidence
17
Evidence (continued) Evidence (continued) CLT ExampleCLT Example
• Teacher’s perception of student learning• Reports
– Structure (could work in groups)• Introduction, Explanation of the 3 examples in the lab
activity, Explain method used, Conclusions
– Results (graded for big picture concepts and conceptual details)• Three “wrong” responses (n=27)
1) “I did not know how to do that [example 4].”2) Mean gets “smaller.”3) An incomplete conclusion.
18
Evidence (continued) Evidence (continued) CLT ExampleCLT Example
• Teacher’s perception of student learning• Exam questions
– Students completed Example 3 (uniform lunch times) from the Course Pack activity on the test• Not previously assigned• No indication that it would be on the test• The “Uniform” subcommand was new• This portion of the test was taken in the lab• Worked individually
19
Test Question OutputTest Question Output
6050403020100
70
60
50
40
30
20
10
0
Sample means n=2
Fre
quen
cy
Variable N Mean StDev
Sample means n=2 1000 29.482 12.232
Sample means n=5 1000 29.926 7.977
Sample means n=20 1000 29.826 3.801
50403020100
100
50
0
Sample means n=5
Fre
quen
cy
403020
100
50
0
Sample means n=20
Fre
quen
cy
20
Evidence (continued) Evidence (continued) CLT ExampleCLT Example
• Teacher’s perception of student learning• Results
– Gave “hint” (minus 1 point) with Uniform command
• Some found Uniform in the pull down menu • Biggest mistake – 1/3 of students used Uniform 30 17.32 instead of Uniform 0 60
– Filled in a table like they had done previously– Wrote a “paragraph” summarizing their findings.
• Mean score 2.6 out of 3 points (st dev = 0.7)
21
Action Research ModelAction Research Model
• What is the problem? I.e., what is not working in the classroom?
• What technique can be used to address the learning problem?
• What type of evidence can be gathered to show whether the implementation is effective?
• What should be done next, based on what was learned?
22
What should be done next, What should be done next, based on what was learned?based on what was learned?
• Improve my writing assignment
• Improve my evaluation of student learning on this concept– Pre/post test– Use delMas/Garfield/Chance instrument (?)– Coordinate with Lunsford on this unit
• Incorporate more report writing in my class
• Keep using and evaluating these activity-based, discovery learning materials.
23
ResourcesResources
• delMas, R., Garfield, J., and Chance B. (1999), A Model of Classroom Research in Action: Developing Simulation Activities to Improve Students' Statistical Reasoning, Journal of Statistics Education v7, n3, http://www.amstat.org/publications/jse/secure/v7n3/delmas.cfm.
• Hollins, E. R. (1999), “Becoming a Reflective Practitioner,” in Pathways to Success in School: Culturally Responsive Teaching, eds. ER Hollins and EI Oiver, Mahwah, NJ: Lawrence Erlbaum Associates.
• Hopkins, D. (1993), A Teacher’s Guide to Classroom Research, Buckingham: Open University Press.
• Noffke, S., and Stevenson, R. (eds.) (1995, Educational Action Research, NY: Teachers College Press.
