100kin10 initiative partner organizations csu stem teacher and researcher (star)* uc berkeley...
TRANSCRIPT
Shared Evaluation: Collaborative Around
Research Experiences for Teachers
Shari LissIndustry Initiatives for Science and Mathematics Education (IISME)
John KellerCalifornia State University STEM Teacher and Researcher Program (STAR)
100Kin10 Initiative
Partner Organizations CSU STEM Teacher and Researcher (STAR)* UC Berkeley CalTeach Summer Research Institute* WMU Experiencing Research for Teaching Science (ExpeRTS)* Industry Initiatives for Science and Mathematics Education (IISME) UofA Teachers in Industry* NCSU Kenan Fellows* *APLU SMTI Campuses (MeetUp facilitated by Charles Coble)
100Kin10 Meetup January 13-14, 2015 IISME Office, Santa Clara, California Day 1: Sharing of program models and evaluation Day 2: Collaborative planning around shared measures of participant and
student outcomes
Collaborative Around Research Experiences for Teachers (^)
CSU STARUC Berkeley CalTeachWMU ExpeRTS
Pre-Service Teacher Researcher Programs
Full-time research at one of 21 lab sites for nine summer weeks Fellows are paid $500/week (most also qualify for relocation support) Weekly education workshops translate lab culture and practices into
classroom Fellows participate in an orientation conference at the beginning of summer Fellows participate in closing conference highlighting their summer
experience Fellows are eligible for professional development support (e.g., travel grants)
StarTeacherResearcher.org
High-Impact Teacher-Researcher Experiences
StarTeacherResearcher.org
States and Agencies Represented
Career Standing
StarTeacherResearcher.org
IISMEUofA Teachers in IndustryNCSU Kenan Fellows
In-Service Teacher Researcher Programs
‘s Purpose
Founded in 1985 industries in partnership with the Lawrence Hall of Science at the University of California at Berkeley.
Transform teaching and learning through industry-education partnerships.
Bringing excitement and relevance to the classroom
What is an Fellowship?Dedicate 90% of time to the Host project and gaining exposure to:Practical applications of subjects they teachNew technologies/skills21st Century Skills
10% Paid Professional Development IISME CoachCommunity Group MeetingsEducation Transfer Plan (ETP)IISME Community Website
What is an Fellowship?
STAR Program Pre-post annual evaluation 2011 pilot longitudinal study 2015 longitudinal study
IISME Pre-post-post annual evaluation 2014 longitudinal study
Prior Evaluation/Research Effort
97% reported being satisfied with overall STAR summer research experience; no Fellows reported being dissatisfied
Of 35 individuals not yet teaching, 27 indicated they planned to pursue K-12 teaching career; other respondents unsure of their future plans
Understanding of science, engineering, and mathematical practices Science Fellows: Planning/carrying out investigations; analyzing and interpreting
data Math Fellows: Use appropriate tools strategically; modeling with mathematics
SUSSI – Nature of Science Creativity and Rationality in Science Diverse Scientific Methods Cultural Embeddedness of Science
STEBI/MTEBI – Teaching Self-Efficacy High scores both pre- and post-summer
StarTeacherResearcher.org
2014 Pre-Post Summer Evaluation
Online survey administered to 176 alumni from 2007-2011116 responses (66%)
Semi-structured interviews of 14 STAR alumni11 in Fall 2011, 7 in Spring 20124 Paired Interviews
StarTeacherResearcher.org
2011 Longitudinal Study of STAR Program
StarTeacherResearcher.org
77% of 2007-11 Alumni on Teaching Career Path
Networking (11% of survey respondents reported that they were in contact with no one) Contact with other Fellows/Master teachers for ideas, lessons,
support
Teacher-Researcher Community Credibility, confidence
Staying connected to researchLearning to do research “like their students”Transfer scientific practices into classroom
Making mistakesNot knowing the answerUsing notebooksPresenting their data to others
StarTeacherResearcher.org
Qualitative Themes
Conclusion of the Summer Fellowship ProgramSatisfaction with Program/Support ServicesImmediate impact on learning and personal growthFeedback for Program improvementMentors complete survey separate satisfaction survey
Sample inquiries: increased understanding of 21st Century Skills gained ideas for making their curriculum more relevant to today’s workplace elevated enthusiasm for teaching increased knowledge of careers that utilize science, math or technology
Annual Comprehensive Surveys
The following school year – Impact ReportETP implementation resultsInfluence Fellowship had on teachingImpact Fellowship had on personal professional development
Sample Inquiries: teaching became more effective implemented or applied new instructional practicesprovided more career information to students increased commitment to teachingexperimented with new teaching strategiesadded industry/research examples and illustrations to teaching
Annual Comprehensive Surveys
Process by Quality Evaluation Designs (QED)
data collected from January through July 2013interviewed 25 randomly selected teachers designed a comprehensive survey for Fellows 2001-2012476 of the 880 teachers who received the email responded
to the survey (54%) interviewed 14 principals
External Evaluation
Overall evaluation results reflect excellent success in all of IISME’s intended outcomes. Nearly 20% of teachers rated the professional development associated with IISME as ‘transformational.’ Highlights of the outcomes include: Strongest impacts Workplace Readiness – increased awareness and integration into classroom of
critical thinking, technology, and teamwork/collaboration skills. Professional Development – enhanced technology knowledge and skills,
understanding of workplace expectations of students, and professional confidence. Teaching Knowledge – greater knowledge of real-world examples and better advising
for students about STEM jobs.
After analyzing the results of the survey, QED reported that they “reflect a highly efficacious program that is consistently well-implemented and strongly valued by teachers and principals.”
External Evaluation - Results
Shared Measures Working GroupsParticipant OutcomesK-12 Student Outcomes
CARET Shared Measures Work
Summer Outcomes MeasuresHow do participants identify STEM practices in their research
experiences that align with learning objectives in their classroom? Post-summer survey item
How do participants design instructional plans that incorporates these particular research practices into their teaching? Rubric for lesson plan analysis
Academic Year Outcomes MeasuresHow do participants implement their instruction plan, with a
focus on these practices? Observation protocol
How do participants reflect on their use of these practices in their instruction? Pre-post lesson survey
How do research experiences influence classroom practices?
Student Perceptions of RET Impact on Teacher and Classroom To your knowledge, did your teacher participate in a summer
research experience? List ways, if any, that you feel this may have made your
class/teacher unique?
Classroom Practices Observed by Students 21st Century Skills NGSS Scientific and Engineering Practices CCSS Mathematical Practices
STEM Career Awareness and Interest Survey Items
Student Outcomes Preliminary Work
Communication and Collaboration Skills
Compared to other STEM classes, . . .how often do you work in collaborative groups ? how often do you get to develop things with other students?how often do you do group projects?How often do you share ideas with other student in groups?How often do you encouraged to share ideas with the whole class?
Critical Thinking/Problem SolvingHow often do you work on problems with more than one right
answer?How often do you use evidence to defend your arguments?
21st Century Skills
Collaborate on Common Research ToolsDraft Questions by end of FebruaryTeleconference mid-March
Meet-up again at APLU SMTI Meeting in JuneRecruit additional collaborative partnersPursue funding to sustain collaborative
NSF Collaborative GrantOther funding
^ Next Steps
What are the shared measures that you care about?What are the measures our funders care about?What should we be collaborating on as a community?
Enough about Us – How about You?
Recommend using pre-existing instruments that are already validated Student evaluation of teachers walks a sticky line; need to be thoughtful/careful with
student comparisons, especially given that students likely don’t know how the teacher taught before summer research experience
Important to also look at level of student engagement in STEM Trajectory of students, involvement in science fairs and other STEM projects
Think about making use of teacher reflections and teacher logs to look for changes in teacher practice
Spend more time defining teaching practices we plan to look for Consider level of confidence in teaching different subjects following RET Many programs still focus on teacher reporting – important to acknowledge teacher self-
reflection and change Funders change their mind quickly and are harder to pin down Useful to look for student enthusiasm for science in addition to engagement Useful to consider softer measures of teacher character, comfort level, coherence for
students – what are predictors for long term outcomes of student success Control and comparison groups – challenges associated, multiple variables to account
Table Shareouts
Contact Information:John Keller, [email protected] Liss, [email protected] Johnson, [email protected]
CARET Website:http://teachersinindustry.arizona.edu/100kin10
Staying in Touch