research on authentic assessment using a virtual world for learning geology
Post on 18-Jan-2016
34 Views
Preview:
DESCRIPTION
TRANSCRIPT
Research on Authentic Assessment Using a Virtual World for Learning Geology
Bernhardt Saini-Eidukat1, Donald Schwert1, Brian Slator2, Lisa Daniels3, and Jeffrey Terpstra4
1Dept. of Geosciences; 2Dept. of Computer Science; 3School of Education; 4Department of Statistics
North Dakota State University, Fargo, ND
NDSU WWWICWorld Wide Web Instructional Committee
Paul Juell
Donald SchwertPhilip McClean
Brian SlatorBernhardt Saini-Eidukat
Alan WhiteJeff Clark
Lisa DanielsJeff Terpstra
WWWIC faculty supported by large teams of undergraduate and graduate students
WWWIC’s virtual worlds research supported by NSF grants DUE-9752548, EAR-9809761, DUE-9981094, ITR-0086142
and EPSCoR 99-77788, and Department of Education FIPSE P116B011528
The Geology Explorer Project
Educational Game designed to provide authentic learn-by-doing experience
•Exploration of a spatially oriented virtual world•Practical, field oriented, expedition planning and
decision making•Scientific problem solving (i.e., a “hands on”
approach to the scientific method)
Balancing Pedagogy with Play
Games have the capacity to engage!
• Powerful mechanisms for instruction
• Illustrate real-world content and structure
• Promote strategic maturity (“learning not the law, but learning to think like a lawyer”)
Technical Approach
• Networked, internet based, client-server simulation
• UNIX-based MOO (Multi-User Dungeon, Object Oriented)
• Java-based clients (text and graphical versions)
The Setting
•Planet Oit - similar to Earth, but opposite the Sun
•Students “land” on Oit to undertake exploration
•Authentic Geoscience goals - e.g., to locate, identify, and report valuable minerals
The Simulation
~50 places: desert, cutbank, cave, etc.
~100 different rocks and minerals
~15 field instruments: rock pick, acid bottle, magnet, etc.
~Software Tutors: agents for equipment, exploration, and deduction
Graphical Navigation
Database Map of Planet Oit
Carrying out Investigations:
Players Identify Outcrops Using Tests
Results of Tests (in this
case, acid reactivity)
Detail Images
Carrying out Investigations:Players Identify Outcrops Using Tests
Because each mineral in the rock can be distinguished, the test can be performed on each mineral. Each returns whether it is acid reactive or not.
Coarse-grained rocks
If the rock contains more than 10% calcite, it fizzes actively when acid is poured on it. If the rock contains between 0 and 10 percent calcite, its powderized form will fizz. Also, if the rock contains more than 25% dolomite, its powder will fizz actively. Finally, if the rock contains no calcite or dolomite, it will not fizz.
Fine-grained rocks
If the property acid_reactive is 2, the mineral fizzes actively when acid is poured on it. If the property acid_reactive is 1, and a powder is made of the mineral, the powder will fizz actively. If the acid_reactive property is 0, the mineral does not fizz.
Minerals
Most other objects in the game have a acid_reactive property of 0. Pouring acid on them will cause them not to fizz. Players are an exception to this rule. You are unable to pour acid on other players.
Other objects
Example of Programming Actions:
Acid Bottle: "pour acid bottle on substance"
Deductive Tutors
• Equipment tutor
• Exploration tutor
• Science tutor
Detects when a student makes a wrong guess and why (i.e. what evidence they are lacking); or when a student makes a correct guess with insufficient evidence (i.e. a lucky guess)
Intelligent Software Tutoring Agents
Students/players interact with the intelligent tutoring agents
Players can ignore advice and carry on at their own risk
Virtual Mapping
SS
LS
SS
Schist
Marble
Qzite
Schist
QtziteDike
Schist Gneiss
LSMarble
Qzite
Qzite
SS
SS Schematic map - not to scale
• Locate yourself and outcrops using topo maps and aerial photos
• General rock identification
• Create a Geologic Map
Virtual Mapping
Tutorials on Concepts of Geologic Mapping are Available in the Environment
A Player (appearance can be changed)
Virtual Mapping
Base Maps for Adding Data and Creating a Geologic Map
“aerial photo”
Virtual Mapping
Base Maps for Adding Data and Creating a Geologic Map
Topographic map
Virtual Mapping
Outcrop Locations are Shown with Markers
Rock Type Markers Show Locations of
Identified Outcrops
You Are Here
Player Creates a Geologic Map Based on Outcrop Locations, and Can Get Immediate Feedback
Virtual Mapping
Virtual Mapping:Formative Assessment
Virtual Mapping:Assessment
Computer Scoring of Your
Map
Rejects the notion of standardized multiple choice tests
Pre-game narrative-based survey• short problem-solving stories • students record their impressions and questions
Similar post-game survey with different but analogous scenarios
Surveys analyzed for improvement in problem-solving
Assessment Using Scenarios: Summative
You are in northern Manitoba on a fishing trip. Fishing has been good. At dawn on the day of your departure you cut across country to a remote bay and have more good luck: you catch your limit of walleye!
Assessment Using Scenarios
On the way back to the lodge, you stumble across a heavy, moss-covered rock on the shoreline, flipping it over. Looking down, you see the underside shines with a brilliant metallic yellow. You pick up the rock and lug it back to the lodge with you.
Assessment Using Scenarios
When it’s time to leave, the pilot notes that you are 45 pounds overweight...exactly the weight of the rock you found. She says, "At $8 a pound, eh, you owe $360."
• As a geologist, what do you do?• List the things you would consider in your decision.• List the questions you would ask yourself, and give
the reasons behind those questions.
Assessment Using Scenarios
Assessment MethodGeneral Problem Solving (60 pts)
1.1 Form a Hypothesis: 20 pts
1.2 Suitable Test: 20 pts
Decide to Test Experimental Design, Comparison, Etc. First Specific Test Second Specific Test 1.3 Conclusions based on evidence: 20 pts
Stated and correct with correct explanation Stated and correct Stated and correct with incorrect explanation Implied and correct Stated and incorrect Just implied Implied and incorrect
?
Specific Information (24 pts)
2.1 Color/Luster: 6 ptsMention testing for Describe how Specific properties
2.2 Malleability/Hardness: 6 pts(ditto) 2.3 Specific Gravity: 6 pts(ditto)
2.4 Other Test : 6 pts(ditto)
Assessment Method
Assessment Method
Expert Knowledge (16)
3.1 Current price of gold: 5 ptsHigh knowledge Some knowledge No knowledge 3.2 Specific Geologic Conditions : 5 pts(ditto)
3.3 Judges prerogative: 6 pts
??
Validation: Multiple Graders Assessment Protocol
Pearson Product Moment Correlation Coefficent between graders
Grader 1 Grader 2 Sample Size Coefficient
jennifert arico 263 0.747
kima arico 263 0.849
kima jennifert 263 0.734
2003 Post Test
0
1020
30
40
5060
70
8090
100
0 20 40 60 80 100
Aric
Kim
A a
nd
Jen
n T
jenn post
kim-a post
Current Assessment Research: Scenario Order; Team vs. Solo play
Students Scen 9/10 Scen 10/9 TotalTeams of 2 19 20 39Solo 162 155 317Total 181 175 356
Preliminary Data from 2004 – scoring is ongoing!
Number of Observations Used Here: 190
Conclusions: Scenario Order; Team vs. Solo play
Scenario Combination Solo Team P-val SignifScen 9/10 9.00 6.83 0.6637 NSScen 10/9 -0.16 11.31 0.0183 S
Net Improvement for 2004
The response variable is Net Improvement based on one grader, “Kelly”.
The data were analyzed as a two-factor factorial design with interaction, based on ordinary least squares methodology.
Geology Explorer Development Team Graphics • Acey Olson (Rendered Landscapes)• Rebecca Potter (Rendered Images, Icons, and Landscapes)• Dan Reetz (Rendered Images, 3D)• Shannon Tomac (Rendered Landscapes, Icons, & Images, Comic Book)• Daniel Turany (Rock/Mineral Photographs)Staff • John Bauer; Java Client (GUMI-bare and GUMI-game) Design and Implementation• Nem Schlecht• Mark Tinguely • Brad Vender; MOO-side HTML server
Students Graduate Research Assistants:
•Kim Addicott; Assessment•Otto Borchert; Software tutors, Java client development •Bob Cosmano; Makes Java Go; Client Implementation•Krista Dauner; Tutoring Scripts •Rahul Devabhaktuni; Hydrology (Water) on Oit •Kuo-Di Jian; Java Tool Integration •Yongxin "George" Jia; Java/LambdaMOO Hierarchy Tool •Guy Hokanson; Software tutors, dynamic HTML •Bhaskar Marthi; Instrument verbs •Vidyalatha Nagareddy; Java/LambdaMOO Entity Tool •Shanhong Wu; Java3D
Undergraduate Research and Independent Study Students: •Bryan Bandli; Assessment and Evaluation •Chris Fish; Endgame Design and Implementation •Kelly Gorz; Assessment and Evaluation•Julia Karst-Grey; Geologic Content and Assessment •Lucas Koehntop; Java consulting •Ned Kruger; Assessment and Evaluation •Acey Olson; Bryce Landscapes, Tutoring, Player History, Interface Design •Carson Rittel; Content Quality Control, Help Database •Joy Turnbull; Content Quality Control, Help Database•Dean Vestal; Content Quality Control, Help Database, Interface Design •Mindy Sue Vogel; Geologic & Quality evaluation, Alternative exercises •K.C. Vorthmann; Geologic Content and Assessment•Jeff Walsh; Geologic Content and Assessment•Jane Willenbring; Assessment and Evaluation
Acknowledgments:Mark Tinguely, who saved our world when its universe imploded, and Dave Schmidt for the name: Planet Oit.
•Aric Olsen; Geologic Content and Assessment•Jennifer Thorstad; Geologic Content and Assessment
http://oit.ndsu.edu
World Wide Web Instructional Committee (WWWIC)
North Dakota State UniversityFargo ND
top related