gk-12 program evaluation: summary of findings · 2019-11-25 · 3. evaluation data sources,...

GK-12 Program Evaluation: Summary of Findings

Annual GK-12 PI Meeting March 28, 2010Beth C. Gamse

111

Overview of Presentation

• Description of Evaluation

• GK- 12 Projects and Participants – GK-12 Fellows – K-12 Teachers

• Comparing GK-12 Fellows to Their Peers

• Former Fellows … After GK-12

• Questions/comments

22

Evaluation Purposes

• To describe the experiences of past and present GK-12 participants, including Fellows and K-12 teachers

• To compare the educational experiences of current GK-12 Fellows with other students in their home departments

• To document the education and early career outcomes of past GK-12 Fellows

33

Evaluation Data Sources, Respondents, and Response Rates

• Surveys for Principal Investigators, GK-12 Fellows, K- 12 Teachers, Faculty Advisors and comparison students, all administered in 2008-09 academic year

• On-line surveys customized for each respondent group

• Response rates:– 81% PIs – 89% current Fellows – 61% current comparison students – 51% former Fellows – 53% Teachers – 50% Faculty Advisors of former Fellows

44

About the Fellowship

• The next set of slides provides data about the GK-12 Fellowship, using data primarily from the Fellows and the Principal Investigators

– Who the Fellows are – Why Fellows apply– How Fellows are prepared– The benefits of participation

5

Graduate Program Discipline, by Fellow Type

4%

4%

4%

6%

7%

8%

10%

15%

27%

6%

5%

3%

8%

7%

7%

8%

17%

28%

0% 5% 10% 15% 20% 25% 30%

Physics

Computer and Information Sciences

Agricultural Sciences/Natural Resources

Mathematics

Geological andEarth Sciences

Ocean/Marine Sciences

Chemistry

Engineering

Biological Sciences

Current FellowsFormer Fellows

Source: Fellows Survey

Back to# 35

66

Criteria Projects Use to Select Fellows

Criteria Percent

Interest in working with K-12 students 96%

Interest in GK-12 fellowship 96%

Interest in conducting outreach to the public 83%

Ability to conduct high quality research 76%

Nomination by faculty member 67%

Prior teaching or K-12 classroom experience 24%

Financial need 8%

Source: PI Survey

77

Fellows’ Reasons for Applying to GK-12

Reasons (Top 6) Former Fellows

Current Fellows

To gain financial support for my education 89% 92%

To develop my teaching skills 73% 83%

To share my knowledge of STEM 68% 80%

To work with school-age students 59% 63%

To have new experiences 55% 66%

To develop my communication skills 53% 80%


8

Gender, by Fellow Type

Former Fellows Current Fellows

Gender

Female54%

Male43%

Chose Not to Report

3%

Female51%

Male46%

Chose Not to Report

3%


9

Race/Ethnicity, by Fellow Type

Race/Ethnicity

White and/or Asian

80% Under- represented

Minority 14%

Chose Note to Report

14%

White and/or Asian

78%

Under- represented

Minority 15%

Chose Note to Report

7%

Former Fellows Current Fellows


1010

Targeted Capacities/Skills of GK- 12 Projects

Target skills/abilities (Top 5) Percent

Ability to teach STEM research and concepts to K-12 students 97%

Ability to generate K-12 students’ interest in STEM research and activities 92%

Ability to develop instructional materials about STEM concepts and methods 83%

Ability to work on a team 78%

Ability to explain STEM research and concepts to public (non-technical) audiences 75%

Source: PI Survey

1111

Topics Covered in Training of GK-12 Fellows

Topics Percent

GK-12 project goals 100%Instructional techniques 95%How to work effectively with teachers 90%Student cognition (how students learn, students’

learning needs and abilities) 90%

How to communicate STEM content 89%How to develop age-appropriate lessons and

activities 88%

Student diversity and equity issues 78%Classroom management strategies 78%Importance of STEM professional involvement in

K-12 education 70%

Source: PI Survey

1212

Common Activities to Prepare Fellows for K-12 Assignments

Activities Former Fellows

Current Fellows

Review GK-12 materials prepared by GK-12 PI, other GK-12 staff 69% 68%

Attended meetings with GK-12 PI and/or other staff 67% 68%

Attended a 1–2 week introductory workshop 64% 74%

Attended an ongoing seminar 58% 54%

Review materials prepared by teachers in participating schools 32% 29%


13

GK-12 Fellows’ School Assignments

1%

1%

28%

31%

38%

1%

1%

11%

31%

55%

0% 10% 20% 30% 40% 50% 60%

An entire school district

A science museum orother educational

organization open to thepublic

Multiple schools

Multiple teachers in asingle school

A single teacher in asingle school

Current FellowsFormer Fellows


14

GK-12 Fellows’ Grade-Level Assignments

0%

21%

47%

42%

1%

31%

47%

53%

0% 10% 20% 30% 40% 50% 60%

Preschool/center-based early child

care

Elementary school

High school

Middle school

Former FellowsCurrent Fellows


1515

Fellows’ Most Common Activities in K-12 Settings

Activities (Top 7) Former Fellows

Current Fellows

Presented lessons 91% 94%Designed new lessons, activities, or

modules 89% 90%

Led small group activities or discussions 82% 84%

Modified existing lessons, activities, or modules 80% 82%

Planned, coordinated, facilitated inquiry-based learning activities 76% 78%

Demonstrated scientific procedures, tools, and techniques 73% 82%

Developed educational resources 64% 59%

Helped K–12 teachers understand and use technology 50% 56%


1616

Benefits of GK-12 Experience to Fellows

GK-12 has…Percent

(who agree or strongly agree)

Increased my interest in outreach to K–12 students or teachers during my career 88%

Helped me to better manage my time 67%

Motivated me to complete my graduate degree 65%

Gave me a deeper understanding of content in my graduate field of study 45%

Led me to consider a career as a K–12 teacher 31%


1717

Faculty Members’ Perceptions of GK-12 Benefits

GK-12 has improved Fellows’ ability to …Percent (who

agree or strongly agree)

Teach STEM to K-12 students 83%

Work on a team 82%

Develop teaching materials for K-12 80%

Facilitate group discussions 79%

Generate K-12 enthusiasm in STEM 75%

Teach STEM to undergraduate students 75%

Source: Faculty Advisor Survey

1818

About Our K-12 Partners

• The next set of slides provides data about the experiences of the K-12 Teachers, using data primarily from the Teachers

– Who the Teachers are – Why Teachers participate – Supports provided to Teachers– The benefits of participation

1919

Teachers’ Experience

Source: Teachers Survey

2%15%

27%56%

1-2 years 3-5 years6-10 years 11-plus years

Years of K-12 Teaching Experience

55%35%

10%

1-2 years3-5 yearsMore than five years

Years Participated in GK-12

2020

Teachers’ Educational Attainment


36%

61%

3%

Bachelors Masters Doctoral

Highest Degree Earned

21

Teachers’ Current Grade Level Assignments

Middle Only35%

(n=258) Elementary Only21%

(n=153)

Not Currently Teaching

3%(n=25)

Multiple Grade Levels

6%(n=41)

High Only35%

(n=254)


2222

Teachers’ Reasons to Participate in GK-12

Reasons Percent

To help my own K-12 students 90%

To deepen my STEM content knowledge 75%

To learn more about mathematics/science research 49%

To help Fellows develop their communication/teaching skills 42%


2323

Projects’ Targeted Topics for K-12 Teachers

Our GK-12 Project is designed to enhance … Percent

Teachers’ STEM content knowledge 97%

Teachers’ skills and techniques for STEM instruction 87%

Teachers’ interest in different instructional strategies 80%

Access to university resources for teachers 81%

Teachers’ participation in professional development about STEM education 80%

Source: PI Survey

2424

Supports GK-12 Provides to Teachers

Types of Support Percent

Opportunities to consult with Fellows for planning 95%

Stipend for participation 92%

Equipment and materials provided to schools 75%

Access to university resources (e.g., library, laboratory, equipment) 73%

Opportunities to present about GK-12 activities at professional conferences 65%

Loaned equipment and materials 64%

Continuing education credits 55%

Stipend for attending professional development workshops and trainings outside GK-12 42%

Source: PI Survey

2525

Contact between Teachers and GK-Fellows


16%

65%

14%5%

Daily or almost daily Once or twice a weekOnce or twice a month A few times a year

262626

Teachers’ Content Knowledge Influenced by GK-12


Topics/Content Knowledge PercentHands-on & manipulative instruction in math & science 37%

What Math/science researchers do 29%

Math/science content you teach 27%

Math/science content more advanced 24%

Current math/science research findings 19%

Math and science technology related instruction 18%

Student misconceptions in math/science 16%

2727

Comparing Current GK-12 Fellows and Their Peers

• The next set of slides provides information about the graduate school experiences of current Fellows in comparison to other current graduate students in the same departments and institutions

– Why graduate students apply for Fellowships – Types of activities in graduate school – Perceived benefits of participation– Volunteer and work activities in graduate school– Funding and other honors/awards– Current career goals

2828

Why GK-12 and Peer Students Applied for a GK-12 Fellowship

To gain financial support for education

To share my knowledge of STEM* (PhD only)

Faculty member encouraged me to apply

To develop my teaching skills

To have new experiences

To work with school-aged students* (PhD only)

To enhance my CV/resume

To develop my communication skills* (PhD only)

To develop my teamwork skills*

Another graduate student encouraged me to apply* (Master’s only)

I was interested in a teaching career*

Source: Fellows Survey* Statistically significant (p<.05)

2929

Differences between GK 12 and Peer Students in Types of Graduate School Activities

Explain STEM concepts to public*

Present work at a professional conference*

Develop instructional materials for undergraduates* (Master’s only)

Generate undergraduate interest in STEM*

Work on a team*

Lead a team*

Facilitate group discussions*

Conduct supervised research* (Master’s only)

Conduct independent research*

Develop research/technological agenda*


3030

Differences between GK-12 and Peer Students about Perceived Benefits of Their Graduate School Education

• Explain general STEM concepts to public audience*

• Teach STEM concepts and methods to K-12 students*

• Develop instructional materials about STEM concepts and methods for K-12 students*

• Generate K-12 interest in STEM activities*

• Teach STEM concepts and methods to undergraduates*


3131

Differences between GK-12 and Peer Students’ Volunteer and Work Activities

• Volunteered in an educational setting with elementary, middle or high school students*

• Volunteered to work with children or youth in a non- educational setting* (PhD students only)

• Worked in an occupation unrelated to my graduate program to earn extra money*


3232

Differences between GK-12 and Peer Students’ Receipt of Funding and Other Honors/Awards

Master’s Fellows GK-12 Non-GK-12

Research award or honor* 25% 15%

Teaching award or honor* 9% 3%

Research funding on own 23% 17%

Doctoral Fellows GK-12 Non-GK-12

Research award or honor 35% 29%

Teaching award or honor* 19% 9%

Research funding on own 34% 28%


3333

GK-Fellows’ Current Career Goals

Master’s Fellows GK-12 Non-GK-12

Primarily research-related employment 65% 66%

Primarily teaching-related employment* 28% 17%

Other employment 43% 44%

Doctoral Fellows GK-12 Non-GK-12

Primarily research-related employment 84% 89%

Primarily teaching-related employment* 51% 35%

Other employment 21% 19%


3434

Former Fellows, After GK-12

• The next set of slides provides data about the experiences of the GK-12 Fellows after they had completed their fellowship experiences

– Degree completion status of former Fellows – What former Fellows do now – Subsequent graduate study for former Master’s students– Perceived benefits of GK-12

35

Current Status of Former Fellows, by Fellowship Degree Level

91%

3%6%

58%

31%

11%

0%

20%

40%

60%

80%

100%

Graduated Still enrolled Left degree program

% Master’s% Doctoral


3636

Former Fellows, After GK-12

• Median time to degree for PhD graduates is 6 years

• No differences in time to degree for female and male students

• 40% of former Master’s Fellows subsequently enrolled in other graduate programs

• 81% of former PhD Fellows and 67% of former Master’s Fellows found jobs within 3 months

• Nearly 75% of former PhD Fellows and 25% of Master’s Fellows work in higher education

37

Primary Discipline Former Fellows Use in Current Work

5%

6%

6%

6%

9%

14%

22%

0% 5% 10% 15% 20% 25%

Ocean/MarineSciences

Science Education

Mathematics

Geological andEarth Sciences

Chemistry

Engineering

Biological Sciences


Go toAll Fellows

38

Former Master’s Fellows Subsequent Graduate Study, by Program Type

Ph.D. 73%

M.A./M.S. 15%

M.D. 4%

J.D. 2%

M.B.A. 2%

Ed.D. 1%

Other3%


39

Former Fellows’ Current Employment Setting, by Degree Level

2%

11%

12%

72%

14%

18%

38%

24%

0% 20% 40% 60% 80%

K-12 school ordistrict

Government

Industry/business

College or university

% Master's% Doctoral


40

Employment of Former Fellows in Government Positions

Federal Dept or Agency

41%

Federal Laboratory

21%

Other7%

Foreign Government

1%Federal

Legislative or Judicial Branch

1%

State or Local Government

29%


4141

Former Fellows’ Perceptions of Program’s Benefits

Experience benefited their ability to… (Top 5) Former Fellows

Explain STEM research and concepts to public audiences 90%

Facilitate group discussions 89%

Work on a team 83%

Lead a team 79%Present own work/my work at a professional

conference 54%


4242

Other Takeaway Findings

• GK-12 experience improved Fellows’ communication, teaching, and teamwork skills

• 90 percent of Fellows believe K-12 teacher—researcher interactions provide reciprocal benefits

• GK-12 Fellows more likely to consider teaching-related employment

• Majority of teachers reported positive effects of GK-12

• Over 80% of PIs plan to sustain GK-12 elements

• Much more detail in the forthcoming Final Report!

43

Next Steps

• Complete final comprehensive report (April)

• Prepare final dissemination report (late spring)

4444

Questions and Comments

• Questions about the evaluation from the audience– What else would be of interest to GK-12 PIs and

participants?– How consistent are national findings with local evaluation

findings?– What would you suggest to NSF about coordination

between local and national evaluation activities?

• How would you think about defining a comparison group?

• How might NSF think about conducting a longer-term follow-up study?

45

GK-12 Evaluation Staff

• Beth Gamse ([email protected]; 617-349-2808)

• Kristen Neishi ([email protected]; 301-634-1759)

• Amanda Parsad ([email protected]; 301-634-1791)

• Carter Smith ([email protected]; 617-349-2543)

• Jennifer Carney ([email protected]; 301-634-1747)

• [email protected]

mailto:[email protected]






gk-12 program evaluation: summary of findings · 2019-11-25 · 3. evaluation data sources,...

Documents