asp2013
TRANSCRIPT
To connect to the internet:
If you have a laptop or tablet and a gmail account visit: http://sjsuriot.appspot.com/
Sign in with your gmail account
We will be using this application during part of the workshop
Don’t have a laptop or tablet? No problem! You can still fully participate.
Welcome!
R.I.O.T. (Real-time Instructor Observing Tool)
Provides meaningful classroom evaluations.
ASP 2013San José State University
Cassandra PaulAndrew Reid
Courses using Interactive-Engagement techniques have
been shown to improve instruction. (Hake 1998; Crouch & Mazur 2001; Prather et al. 2009; and many others)
Many classroom observation protocols are developed for researchers/evaluators to use, and not for instructors. (Examples: Active-Learning Inventory Tool, Amburgh et al. 2007; Reformed Teaching Observation Protocol, Sawada et al. 2002)
A poor ‘score’ on a protocol can cause a negative overall evaluation experience for the instructor.
The UC Davis Physics Education Research group wanted to make an observing tool that: Was easy for instructors to use
Gave instructors lots of data on their classroom
Let instructors make their own judgments on how to change
Motivation
Real-time Instructor Observing Tool
UC Davis team developed RIOT
Categories created for easy coding
Instructors see a visual representation of data collected in their classroom
No video
RIOT data can be discussed in an professional development environment, consultation or private reflection
Clarifying Instructions
Explaining Physics
Listening to Question
Closed Dialogue w/students
Open Dialogue w/ students
Open Dialogue with Ideas
Passively Observing Students
Actively Observing Students
Checking Homework
Out of Room
Fixing Apparatus
Not Interacting/Reading
Chatting with Students*West, E. A., Paul, C. A., Potter, W. H., Webb, D. Variation of instructor-student interactions in an introductory interactive physics course Phys. Rev. ST Phys. Educ. Res., Vol. 9 (March 2013)
Computerized Classroom Observing Protocol
Observer watches class
Clicks icons pertaining to interaction taking place
Report is auto-generated at the end
Designed to be used by and for instructors to inform teaching
Interaction categories are (relatively) easy to interpret
Initially developed for Physics TAs at UC Davis
Also used as research tool*
Little training required
After today you should be ready to go!
(More training required if using RIOT for research purposes.)
Real-time Instructor Observing Tool
*West, E. A., Paul, C. A., Potter, W. H., Webb, D. Variation of instructor-student interactions in an introductory interactive physics course Phys. Rev. ST Phys. Educ. Res., Vol. 9 (March 2013)
Outline
Intro
Using RIOT:
How to make observations
How to download and view results
Interpreting RIOT reports:
Small groups: Look at example data from courses
Whole class discussion: group findings and applications
Identifying Interactions:
Practice coding
Using RIOT
http://sjsuriot.appspot.com/
R.I.O.T Application
Clarifying Instructions
Explaining Physics
Listening to Question
Closed Dialogue w/students
Open Dialogue w/ students
Open Dialogue with Ideas
Passively Observing Students
Actively Observing Students
Checking Homework
Out of Room
Fixing Apparatus
Not Interacting/Reading
Chatting with StudentsTime in minutes
TA is Interacting w/ Whole Class
TA is interacting w/ Group 1
TA is interacting w/ Group 2
TA is not Interacting
Sum of all Group rows
TA is interacting with individual
R.I.O.T. OUTPUT EXPLAINED BY ROW
TA is interacting w/ Whole Class during time when students are in small groups
Interpreting RIOT Reports
CLASPCOLLABORATIVE LEARNING THROUGH ACTIVE SENSE-MAKING IN PHYSICS
2 pieces of
CLASP
curriculum
Time spent
in class per
week:
Interactivity: Number of
Students:
Instructors:
Lecture 1x 80 minutes
(25 min Quiz)
(sometimes)
Peer-Instruction
~150
students (
x2 per
course)
Usually faculty,
sometimes
lecturer or
advanced grad
Discussion-
Lab
2 x 140
minutes
Series of
interactive
activities spliced
with whole class
discussions
25-30
students
(x11 per
course)
The vast
majority are
grad students.
Sodium7A DLM 17 3/12/2008
Silicon7A DLM 17 3/11/2008
Clarifying Instructions
Explaining Physics
Listening to Question
Closed Dialogue w/students
Open Dialogue w/ students
Open Dialogue with Ideas
Passively Observing Students
Actively Observing Students
Checking Homework
Out of Room
Fixing Apparatus
Not Interacting/Reading
Chatting with Students
WCSG
SGWC
Lithium7B DLM 3 1/15/08
Titanium7B DLM 3 1/14/08
Identifying Interactions
Dr. Tom Fleming – University of Arizona
intro astronomy class
http://www.youtube.com/watch?v=LIa5hjVAAD8&list=PL19CE26ACB414B35B
Lecture Clip
Learning Assistant Resource Videos
University of Colorado
Lab/Discussion Clip
Benefits of RIOT
Coarse measurement allows for lots of observations
Output gives you an illustrative view of classroom (you can learn a lot of things about the instructor in seconds, our eyes respond to patterns) Excellent TA training tool
Not as invasive as video tape
instructors more likely to allow (invite!) observations (objective & anonymous)
Easier to act naturally in front of
Students not video taped (IRB exempt)
Can be modified to measure MANY things
Instantly turns qualitative data into quantitative data for statistical analysis
Weaknesses of RIOT
Not a replacement for video tape
Coarse observation
Info on quality lost (in current form)
Only gives you info regarding what TA is doing (student
info is lost)
You can’t go back and re-analyze interactions
(Next step to see if Active Observing is a true indicator for
student achievement is to see what happens before and after
active observing)
Future uses for RIOT
Professional development tool (faculty love data!)
Organizer for field notes (new feature!)
Small Group analysis (identifying common patterns in
student discourse)
Look for issues with the curriculum (for example: places
where instructors explain a lot)
As a ‘tag’ for videotape data
PRIME grant: Student Participation Observing Tool
(SPOT)
Thank you!
Cassandra [email protected]
Andrew [email protected]
RIOT: sjsuriot.appspot.com/
Slides and more info at: www.sjsu.edu/people/cassandra.paul/RIOT/
Extras
24
Talking AtStudents
Talking WithStudents
Observing Students CLASP A Observation 1
25
Talking AtStudents
Talking WithStudents
Observing Students CLASP A Observation 1
26
Talking AtStudents
Talking WithStudents
Observing Students CLASP A Observation 1
27
Talking AtStudents
Talking WithStudents
Observing Students CLASP A Observation 1
28
Talking AtStudents
Talking WithStudents
Observing Students CLASP A Observation 1
SMALL GROUP TIME
29
Instructors spend between 5% and 50% of their small
class time not interacting with their students.
Instructors spend between 1% and 75% of their whole
class discussion time explaining to their students.
WHOLE CLASS TIME
30
RIOT Findings
There is a large variation in the range of instructor-
student interactions between instructors
THE AMOUNT OF TIME
SPENT IN WHOLE CLASS
DISCUSSION VARIES IN
EACH OF THE THREE
SEGMENTS OF THE COURSE.
THE REGULARITY OF
CERTAIN
INTERACTIONS, LIKE
DIALOGUING, VARIES IN
EACH OF THE THREE
SEGMENTS OF THE COURSE
Carb
on 7
C 2
008
Carb
on 7
B 2
009
RIOT Findings
There is a large variation in the range of instructor-
student interactions between instructors
The curriculum affects how instructors spend their
time in class.
By changing the materials, developers can affect the
amount of time instructors spend interacting in
certain ways
Do interactions have impact on student
achievement?
RIOT Findings
There is a large variation in the range of instructor-student interactions between instructors
The curriculum affects how instructors spend their time in class. By changing the materials, developers can affect the amount
of time instructors spend interacting in certain ways
There is evidence that suggests that certain instructor interactions are correlated with student achievement (active observing in particular)
Student Participation
Observing Tool
39