rebecca stanfield mccown daniel laven nora mitchell

13
Early Attempts to Build Early Attempts to Build Evaluation Capacity: Evaluation Capacity: Examples from National Park Examples from National Park Service Youth Engagement Service Youth Engagement Programs Programs Rebecca Stanfield McCown Rebecca Stanfield McCown Daniel Laven Daniel Laven Nora Mitchell Nora Mitchell National Park Service Conservation Study National Park Service Conservation Study Institute Institute Jennifer Jewiss Jennifer Jewiss University of Vermont University of Vermont

Upload: tomas

Post on 01-Feb-2016

38 views

Category:

Documents


0 download

DESCRIPTION

Early Attempts to Build Evaluation Capacity: Examples from National Park Service Youth Engagement Programs. Rebecca Stanfield McCown Daniel Laven Nora Mitchell National Park Service Conservation Study Institute Jennifer Jewiss University of Vermont. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Rebecca Stanfield McCown Daniel Laven Nora Mitchell

Early Attempts to Build Early Attempts to Build Evaluation Capacity: Evaluation Capacity:

Examples from National Park Examples from National Park Service Youth Engagement Service Youth Engagement

ProgramsProgramsRebecca Stanfield McCownRebecca Stanfield McCown

Daniel LavenDaniel Laven

Nora MitchellNora Mitchell

National Park Service Conservation Study National Park Service Conservation Study InstituteInstitute

Jennifer JewissJennifer Jewiss

University of VermontUniversity of Vermont

Page 2: Rebecca Stanfield McCown Daniel Laven Nora Mitchell

The need for evaluation The need for evaluation research of diversity research of diversity programs in the NPSprograms in the NPS Significant research has been done to Significant research has been done to

articulate the issue and identify barriers with articulate the issue and identify barriers with regards to diversity in the national parksregards to diversity in the national parks Theories for underrepresentation (Floyd 1999; Theories for underrepresentation (Floyd 1999;

Gomez 2002; Johnson, Bowker, English, and Gomez 2002; Johnson, Bowker, English, and Worthen 1998)Worthen 1998)

Identified barriers to visitation (Payne, Mowen, Identified barriers to visitation (Payne, Mowen, and Orsega-Smith 2002; Solop, Hagen, and and Orsega-Smith 2002; Solop, Hagen, and Ostergre, 2003; Tinsely, Tinsley, and Croskeys Ostergre, 2003; Tinsely, Tinsley, and Croskeys 2002)2002)

Page 3: Rebecca Stanfield McCown Daniel Laven Nora Mitchell

The need for evaluation The need for evaluation research of diversity research of diversity programs in the NPSprograms in the NPS

Few (if any) studies have examined cases Few (if any) studies have examined cases where the NPS has been successful in where the NPS has been successful in addressing issues of diversity and engaging addressing issues of diversity and engaging traditionally underserved audiencestraditionally underserved audiences

Page 4: Rebecca Stanfield McCown Daniel Laven Nora Mitchell

Keeping National Parks Keeping National Parks Relevant in the 21Relevant in the 21stst Century Century Focus areas:Focus areas:

- - working with others to tell working with others to tell inclusive stories;inclusive stories;

- engage in an ongoing - engage in an ongoing dialogue with openness, dialogue with openness, sensitivity, and honesty;sensitivity, and honesty;

- sustain community - sustain community relationships;relationships;

- create a workforce - create a workforce reflective of society reflective of society

((Mitchell, Morrison, Farley, and Walter 2006Mitchell, Morrison, Farley, and Walter 2006))

Page 5: Rebecca Stanfield McCown Daniel Laven Nora Mitchell

Using evaluation research to Using evaluation research to understand diversity understand diversity programs in the NPSprograms in the NPS

Evaluation can provide a systematic Evaluation can provide a systematic assessment of a program or policy, contribute assessment of a program or policy, contribute to its improvements, prioritize resources, and to its improvements, prioritize resources, and demonstrate effectiveness demonstrate effectiveness

(adapted from Weiss, 1998)(adapted from Weiss, 1998)

Page 6: Rebecca Stanfield McCown Daniel Laven Nora Mitchell

Using evaluation research to Using evaluation research to understand diversity understand diversity programs in the NPSprograms in the NPS

Evaluation research can help identify barriers, Evaluation research can help identify barriers, opportunities, and strategies for increasing the opportunities, and strategies for increasing the visitation and participation of new and diverse visitation and participation of new and diverse audiences in national parksaudiences in national parks

Evaluation can help inform ongoing program Evaluation can help inform ongoing program designdesign

Page 7: Rebecca Stanfield McCown Daniel Laven Nora Mitchell

Overarching Research and Overarching Research and Evaluation QuestionsEvaluation Questions How do programs overcome barriers and How do programs overcome barriers and

challenges with regards to increasing diversity?challenges with regards to increasing diversity? What are some key ingredients for successfully What are some key ingredients for successfully

engaging diverse youth?engaging diverse youth?

Provide a foundation in the development of an Provide a foundation in the development of an evaluation tool for diversity programsevaluation tool for diversity programs

Begin to build a theory of change model Begin to build a theory of change model regarding diversity programsregarding diversity programs

Page 8: Rebecca Stanfield McCown Daniel Laven Nora Mitchell

Study MethodsStudy Methods

Qualitative interviews with key informants and Qualitative interviews with key informants and stakeholdersstakeholders

Phase 1: Preliminary Interviews and Model Phase 1: Preliminary Interviews and Model DevelopmentDevelopment Interviews with NPS personnel, partners, Interviews with NPS personnel, partners,

academics, and other key informantsacademics, and other key informants

Page 9: Rebecca Stanfield McCown Daniel Laven Nora Mitchell

Study MethodsStudy Methods

Phase 2: Case Study ResearchPhase 2: Case Study Research Youth engagement programs at Santa Monica Youth engagement programs at Santa Monica

Mountains National Recreation Area and Boston Mountains National Recreation Area and Boston Harbor Islands National Park AreaHarbor Islands National Park Area

Examined 7 programs run by the park and Examined 7 programs run by the park and partner organizationspartner organizations

Page 10: Rebecca Stanfield McCown Daniel Laven Nora Mitchell

What we have learned…What we have learned…

Page 11: Rebecca Stanfield McCown Daniel Laven Nora Mitchell

What we have learned…What we have learned…

Cyclical nature of the Cyclical nature of the model model Need to address all areas Need to address all areas

of the model in order to of the model in order to effectively address effectively address diversity issuesdiversity issues

Implications for Phase 2Implications for Phase 2 Building off the model Building off the model

developed in Phase 1developed in Phase 1 Examine how the model Examine how the model

is made operational at is made operational at the field levelthe field level

Page 12: Rebecca Stanfield McCown Daniel Laven Nora Mitchell

What we have learned…What we have learned…

What is the larger context of the program What is the larger context of the program regarding participants’ daily lives?regarding participants’ daily lives? How does the program manage challenges How does the program manage challenges

within participants lives that impact program within participants lives that impact program success?success?

Important considerations:Important considerations: Mentorship within the context of the program Mentorship within the context of the program

and with regards to schooland with regards to school Continued contact throughout the school yearContinued contact throughout the school year Barriers to participation such as access to park Barriers to participation such as access to park

areasareas

Page 13: Rebecca Stanfield McCown Daniel Laven Nora Mitchell

Applying what we have Applying what we have learned to new researchlearned to new research Sharing information throughout the process Sharing information throughout the process

with stakeholders and interested parks and with stakeholders and interested parks and program managersprogram managers

Using evaluation to inform program designUsing evaluation to inform program design SAMO Youth Program informs the development SAMO Youth Program informs the development

of the YIP program in Massachusetts Area Parksof the YIP program in Massachusetts Area Parks Utilize similar study techniquesUtilize similar study techniques

Stakeholder meeting to key ingredients and Stakeholder meeting to key ingredients and program logicprogram logic

Key informant interviewsKey informant interviews