westat’s evaluation of the sig program: findings from the leadership study

AlisonLangham@westat.com1

Westat’s Evaluation of the SIG Program:Findings from the Leadership Study

Alison Langham, Westat AlisonLangham@westat.com

Presentation to the SIGnetwork SPDG Directors’ SessionApril 16, 2010

Project Funded by:Office of Special Education Programs

U.S. Department of Education

AlisonLangham@westat.com2

Presentation Outline

Overview of the SIG Program Overview of Westat’s Evaluation of the SIG

Program Brief explanation of the Leadership Study Results from the Leadership Study Summary and implications

AlisonLangham@westat.com3

Background on the State Improvement (SIG) Program

IDEA ‘97 Joint effort of the feds and states to improve

results for children with disabilities Money to states on a competitive basis Professional development, information

dissemination, TA…SYSTEMIC CHANGE

AlisonLangham@westat.com4

Background on the SIG Program Evaluation

Commissioned by OSEP in 2000 Five-year cooperative agreement, that continued for six years Formative Focused on the overall SIG Program, not individual states—but

individual SIG projects were the unit of analysis Not intended to provide technical assistance to projects Instead, intended to describe what states were doing to inform

OSEP, and the states themselves

AlisonLangham@westat.com5

Background on the SIG Program Evaluation

Premises of the SIG Program that guided our evaluationo Systems change is necessary to effect significant statewide

improvementso Improving systems requires comprehensive planning that

involves multiple individuals, agencies, and institutionso SEAs need to play a leadership role in engaging partners

and bringing about the systems changes

AlisonLangham@westat.com6

Multiple evaluation activitieso Logic Modelso Cross-state comparisonso Systemic Evaluation Inquiry Modelo Model of Theory of Systemic Changeo Outcomes Studyo Administrative Leadership Study

Background on the SIG Program Evaluation

AlisonLangham@westat.com7

Administrative Leadership Study: Framework

SIG Project Administrative

Leadership

Influence Authority Accountability

AlisonLangham@westat.com8

Administrative Leadership Study: Methodology

Nine states Iterative data collection and analysis Telephone interviews and document reviews Qualitative coding

AlisonLangham@westat.com9

Administrative Leadership Study: Results

Influence– Persuasion– Guidance

Authority– Use of position or title– Control of funds

Accountability– Setting expectations– Establishing standards of performance

AlisonLangham@westat.com10

Administrative Leadership Study: Results

Influence StrategiesStrategy A: Implementing professional

development (PD) as a systemic tool for changeStrategy B: Communicating a vision of changeStrategy C: Creating local buy-inStrategy D: Integrating PD efforts with general

education reformStrategy E: Using collaborative working

partnerships that promote joint learning

AlisonLangham@westat.com11

Administrative Leadership Study: Results

    Administrative Leadership Strategies  

    Influence TOTAL

Cohort State A B C D E  

FY1999 a   4

FY1999 b       2

FY1999 c   4

FY1999 d   4

FY1999 e 5

FY1999 f     3

FY2000 g 5

FY2000 h   4

FY2001 i   4

TOTAL 6 6 6 7 9  

AlisonLangham@westat.com12

Administrative Leadership Study: Results

Authority StrategiesStrategy F: Asserting delegated authority

Strategy G: Setting direction

Strategy H: Justifying actions by evoking a higher authority

Strategy I: Providing support for partners and subgrantees who buy-in

AlisonLangham@westat.com13

Administrative Leadership Study: Results

Cohort State

Administrative Leadership Strategies  

Authority

TOTALF G H I

FY1999 a   3

FY1999 b     2

FY1999 c       1

FY1999 d 4

FY1999 e 4

FY1999 f     2

FY2000 g     2

FY2000 h   3

FY2001 i 4

TOTAL 8 8 4 5  

AlisonLangham@westat.com14

Administrative Leadership Study: Results

Accountability StrategiesStrategy J: Standardizing the evaluation

methodology

Strategy K: Prioritizing the tasks associated with accountability

Strategy L: Using accountability data to justify changes in project

AlisonLangham@westat.com15

Administrative Leadership Study: Results

Cohort State

Administrative Leadership Strategies  

Accountability TOTAL

J K L  

FY1999 a 3

FY1999 b 3

FY1999 c       0

FY1999 d   2

FY1999 e   2

FY1999 f     1

FY2000 g   2

FY2000 h     1

FY2001 i 3

TOTAL 6 6 5  

AlisonLangham@westat.com16

Administrative Leadership Study: Summary and Implications

Administrative Leadership Study Findings: Every SIG director used multiple leadership

strategies SIG directors used influence strategies most

and accountability strategies least Awareness of self, Project/Program, current

events and culture influenced which strategies SIG directors used

AlisonLangham@westat.com17

Administrative Leadership Study: Summary and Implications

State culture

Current events

Project and Program

SelfLevels of Awareness

AlisonLangham@westat.com18

Administrative Leadership Study: Summary and Implications

Most important SIG Evaluation finding: strong leadership coincided with strong SIG project outcomes

Implication: leadership will matter in SPDG projects, too