1
Missouri Program
11
gImprovement Initiative
Presented by Lennox McLendon & Kathi Polis
Quality First
It is all about
Program Improvement!
Getting Better Every Year
22McLendon & Polis 2008McLendon & Polis 2008
at Serving Adult Learners
How did we get here?
Pre-1998
Pre-1998National Literacy Act of 1991Adult Education was a ‘regulatory’ program
5%, 10%, 15%, 20%
33McLendon & Polis 2008McLendon & Polis 2008
Direction from Washington regarding how to run our state and local programs
1998
Congress listenedMissouri is different from New Hampshire.Columbia is different from Kansas City.Program decisions should be made as l t th l ibl
44McLendon & Polis 2008McLendon & Polis 2008
close to the learner as possible.Limit regulationsGrant flexibility
BFO
55McLendon & Polis 2008McLendon & Polis 2008
Post-1998
Performance MeasuresEducational gainsEmploymentSecondary credentialsTransition to postsecondary
66McLendon & Polis 2008McLendon & Polis 2008
DRAW A LINEStart where you are now and get better every year.
2
First Crisis!
Performance without DataSix years getting data systems to measure performance
Assessment policiesAssessmentsT i i t
77McLendon & Polis 2008McLendon & Polis 2008
Training everyone to follow protocolsTraining everyone to enter the data correctlyTrying to understand the reports
Second Crisis!
88McLendon & Polis 2008McLendon & Polis 2008
Getting better every year!
Second Crisis
HTHDWDT
99McLendon & Polis 2008McLendon & Polis 2008
Regulations are Easier
Data Data Everywhere!
Now What Do We Do?
Most StatesFind and fix!
1010McLendon & Polis 2008McLendon & Polis 2008
The Missouri Advantage
Missouri Advantage
Critical Success FactorsWhat is important to your program?
If there is a concern, what are some of the possible causes?What are some probing questions we could ask to narrow the causes?
1111McLendon & Polis 2008McLendon & Polis 2008
to narrow the causes?What are some possible solutions?You are the first to have this resource!
Objectives for Day One
Review the critical success factors and statewide needs
Review the initiatives the state staff have underway to address the statewide needs
1212McLendon & Polis 2008McLendon & Polis 2008
statewide needs
Review the roles and responsibilities of the Program Improvement Team
3
Objectives for Day 1
Examine locally-identified needs – the Red Flag Reports
Clarify the guidelines and timelines for submission of PI plans
1313McLendon & Polis 2008McLendon & Polis 2008
Review the program improvement process, tools, and strategies
Objectives for Day 2
Learner PersistenceExamine the latest research on student persistence.Determine research implications for the delivery of adult education services.E l i t f i t ti l d
1414McLendon & Polis 2008McLendon & Polis 2008
Explore a variety of instructional and management strategies.Examine professional development options that can help instructors support student persistence.Review a template for a student persistence learning project.
Challenges
1. Until now, there has been no collection of the evidence-based strategies.
2. We all come to adult education sideways!
1515McLendon & Polis 2008McLendon & Polis 2008
It is all about Program Improvement!
There is no front door!
1616McLendon & Polis 2008McLendon & Polis 2008
How did YOU get here?
Activity 1
At your tables, share how you got into adult education.
1717McLendon & Polis 2008McLendon & Polis 2008
Two Guiding Principles
The Trident
The Wisdom of Crowds
1818McLendon & Polis 2008McLendon & Polis 2008
4
Pg. 5Pg. 5
1919McLendon & Polis 2008McLendon & Polis 2008
Wisdom of Crowds
A group of informed individuals give better advice and wiser recommendations than experts.
People will not argue with things they help develop.
2020McLendon & Polis 2008McLendon & Polis 2008
g y p p
Advice: Involve as many of your staff as possible in program improvement activities.
Objective #1
Review the critical success factors andstatewide needs.
2121McLendon & Polis 2008McLendon & Polis 2008
statewide needs.
Critical Success Factors
Are students coming?
Are students staying?
A t d t l i ?
2222McLendon & Polis 2008McLendon & Polis 2008
Are students learning?
Are Students Coming?
Critical Success FactorsThree year trends on enrollment
Three year trends on contact hours
2323McLendon & Polis 2008McLendon & Polis 2008
Are Students Staying?
70% of enrollees attend at least 12 hours and are pre-tested.
55% of the pre-tested students will be post-tested.
Average contact hours as compared to state average h f
2424McLendon & Polis 2008McLendon & Polis 2008
contact hours for:ABEASEESOLCorrections
5
Are Students Learning?
Performance MeasuresEducational gains
High school credentials
2525McLendon & Polis 2008McLendon & Polis 2008
Transitions to postsecondary
Employment
Job retention
Statewide Needs
Early Dropouts
Student Persistence
2626McLendon & Polis 2008McLendon & Polis 2008
Secondary Credentials
Transition to Postsecondary
Objective 2
Review the initiatives the state staff have underway to address the statewide needs.
2727McLendon & Polis 2008McLendon & Polis 2008
State Program Improvement Initiatives
Early Dropouts
Managed Intake and EnrollmentIdentify managed intake models and build professional development around them
2828McLendon & Polis 2008McLendon & Polis 2008
around them.Analyze data from programs using managed enrollment to document its impact.Develop pilot sites for managed enrollment.
State Program Improvement Initiatives
Student Persistence
Statewide/regional learner persistence workshops and resources
Support for local programs to conduct their own—conducted by the PIT membersAdding the training to the PDC menu
2929McLendon & Polis 2008McLendon & Polis 2008
Promoting NCSALL’s study circles for local programsDeveloping a virtual library of persistence resources
Clear policy (guidance and expectations) so that every practitioner is clear on what is expected and how to fulfill those expectations
State Program Improvement Initiatives
Secondary Credentials
Clear guidance/policies on appropriate GED goal setting Training for all appropriate
titi GED l tti
3030McLendon & Polis 2008McLendon & Polis 2008
practitioners on GED goal settingIdentification of local program (in-state and out-of-state) models and promising practices for GED completions
6
State Program Improvement Initiatives
Transition to Postsecondary
Continue work of Transitions TaskforceQuality indicators for college transition to provide clear guidance
3131McLendon & Polis 2008McLendon & Polis 2008
transition to provide clear guidance Guidance/policies on appropriate postsecondary goal setting Training for all appropriate practitioners on postsecondary goal setting Crosswalk between TABE and college entrance exams
Ongoing State Office Initiatives
Review of Selected State InitiativesContent standardsProfessional developmentLearning difficultiesGED OnlineContinuous Quality Improvement
3232McLendon & Polis 2008McLendon & Polis 2008
Continuous Quality ImprovementSupplemental LiteracyPartnerships/workforce development pilotMarketing
Ongoing State Office InitiativesReview current:
PoliciesProductsProfessional DevelopmentPractice
Interview state and local practitioners.Query other states
3333McLendon & Polis 2008McLendon & Polis 2008
Query other states.September 30 report will include:
An analysis of each initiativePositive componentsGaps and/or weaknessesRecommendations and sources for alternative strategies and options
State Staff Professional Development
Electronic SurveyUsing evidence-based practicesFacilitating workgroups Developing policy that provides clear guidance and expectationsCommunicating continuity, sequence and I t ti ( “d j ’)
3434McLendon & Polis 2008McLendon & Polis 2008
Integration (no “du jour’)Overseeing pilot tests (no “Gut and Guru”)Measuring impactIntervening in low-performing programs
Staff-wide NeedsIndividual Needs
Objective 3
Review the roles and responsibilities of the Program Improvement Teams.
3535McLendon & Polis 2008McLendon & Polis 2008
Why are you here?Why are you here?
The PIT Crew
Wisdom of Crowds
A team approach
Different perspectives
3636McLendon & Polis 2008McLendon & Polis 2008
Representative of your colleagues
7
The PIT Crew
Your missionYour missionTo regularly communicate with all staff to get them engaged in the PI process
Remember your representative role.To analyze program data for the purposes of
l i d i l ti i ifi t
3737McLendon & Polis 2008McLendon & Polis 2008
planning and implementing significant improvements within your programTo track and measure the impact of your PI effortsTo submit PI planning documents to state office
Activity 2 – What do you believe?
What makes an effective Program Improvement Team?
With your PIT crew, make a list of what you
3838McLendon & Polis 2008McLendon & Polis 2008
y , ybelieve will be important things for the crew to remember, do, or feel to guide your PI process.
Example: We believe that:All staff should have a voice in the PI process.Adult learners have much to contribute to our PI
process.
Objective 4
Examine and clarify locally-identified needs.
3939McLendon & Polis 2008McLendon & Polis 2008
Structure of the Reports
Critical Success Factors
Possible Causes
Probing Questions
4040McLendon & Polis 2008McLendon & Polis 2008
Possible Strategies
The Wisdom of Crowds
Activity 3 – Examining the Structure
With your PIT crew, examine the structure of the report.
Make a list of any overall questions or concerns that you have.
4141McLendon & Polis 2008McLendon & Polis 2008
Reviewing Your Program Reports
Activity 4 – PIT Crew WorkSelect ONE Red FlagKeep notes from the meeting with your ‘Crowd’
Other critical success factors
4242McLendon & Polis 2008McLendon & Polis 2008
Other causesAdditional helpful probing questionsMOST IMPORTANTLY, other strategies that might work.
8
Group Discussion
What INSIGHTS did the analysis have for you?
What is still NOT CLEAR?
4343McLendon & Polis 2008McLendon & Polis 2008
Objective 5
Clarify the guidelines and timelines for submission of PI plans.
4444McLendon & Polis 2008McLendon & Polis 2008
The Big Picture
Preliminary Planning DocumentDue September 15, 2008.
Program Improvement PlansDue November 1, 2008
Implementation of Plan
4545McLendon & Polis 2008McLendon & Polis 2008
December, 2008 – May, 2009Progress Report
June 15, 2009
The Big Picture
Preliminary Planning DocumentDue September 15, 2008
[email protected] steps
How, when, who?How you will:
Review performance results with staff
Pg. 7Pg. 7
4646McLendon & Polis 2008McLendon & Polis 2008
Review performance results with staffPrioritize needsSet a vision, goals, and performance indicatorsSelect a strategy or approach to pilotPlan and implement the pilotGo to scale and measure impact
The Big Picture
Program Improvement PlanDue November 1, 2008Program profileNeeds assessment and visionGoals and performance indicators
Pg. 10Pg. 10
4747McLendon & Polis 2008McLendon & Polis 2008
Strategies and researchActivities and timelineEvaluation
Objective 6
Review the program improvement process, tools, and strategies.
4848McLendon & Polis 2008McLendon & Polis 2008
9
Getting Started
How do we use the Red Flag reports?How do we engage our fellow staff members in identifying what we need to fix first?How do we pilot test new strategies?H d i l t th
4949McLendon & Polis 2008McLendon & Polis 2008
How do we implement the new strategies throughout our program?
Three Resources
The Trident
Electronic powerpoints and handouts from today’s workshop for you to adapt
5050McLendon & Polis 2008McLendon & Polis 2008
Going to Scale A step by step guide to program improvement
Pg. 16Pg. 16
The Steps
1. Identifying and prioritizing your program improvement needs
2. Narrowing the root cause using probing questions
3. Setting a vision, goals, and evidence4 L ti d l ti t t i
5151McLendon & Polis 2008McLendon & Polis 2008
4. Locating and selecting strategies5. Planning and implementing a pilot
test6. Measuring impact7. Tweaking and going to scale
Identifying and prioritizing your program improvement needs
Identifying program improvement needs
The Red Flag reportConsider what the state office is developing and perhaps eliminate those for now
5252McLendon & Polis 2008McLendon & Polis 2008
for now.
Identifying and prioritizing your program improvement needs
Prioritizing which needs to attack firstEngage as many of your staff as possible.
Ask them “How much impact would each have on our program if we could fix it?” 0=minimal impact;
5353McLendon & Polis 2008McLendon & Polis 2008
could fix it? 0=minimal impact; 4=major impact
Electronic surveyStaff meeting
Or put each need on a sheet and give each staff member two sticky dots.
Prioritize the top one – two needs you will address this year.
Narrowing the root cause using probing questions
What is causing this part of our program not to work?
Engage as many of the staff as possible.Use the possible causes and probing questions in the Red Flag report to
5454McLendon & Polis 2008McLendon & Polis 2008
questions in the Red Flag report to narrow down the causes.You may not be able to isolate it, but the discussion is helpful to understand and clarify the issue.
10
Setting a vision, goals, and evidence
For each priority, ask the question:“What would this look like if it were
working perfectly?”
Staff members will have different i i
5555McLendon & Polis 2008McLendon & Polis 2008
visions.Blend the different visions.
A common vision makes finding new strategies much easier.
Setting a vision, goals, and evidence
Goals: To reach this vision, we will need to accomplish this…1. …2. …
5656McLendon & Polis 2008McLendon & Polis 2008
3. …4. …
Setting a vision, goals, and evidence
Evidence--Ask each staff member“If we pilot test a new strategy to solve this problem, what evidencewhat evidence will you need to be convinced it works?”
P f I di t
5757McLendon & Polis 2008McLendon & Polis 2008
Performance IndicatorsFor each goal, include an objective, measurable method for measuring attainment.
Locating and selecting strategies
You now haveA prioritized needAn understanding of the causeA unified vision of what it will look like when it is working perfectly, and
5858McLendon & Polis 2008McLendon & Polis 2008
Evidence you will need to collect to satisfy all staff members.
Red Flag reportsNow you can go look for other strategies!
CAELACAELA
Center for Adult English Language Acquisition at the Center for Applied Linguistics.
http://www.cal.org/caela/research/
5959McLendon & Polis 2008McLendon & Polis 2008
6060McLendon & Polis 2008McLendon & Polis 2008
11
6161McLendon & Polis 2008McLendon & Polis 2008
TESOLTESOL
Teachers of English to Speakers of Other Languages
www.tesol.org
6262McLendon & Polis 2008McLendon & Polis 2008
professional issues > research
6363McLendon & Polis 2008McLendon & Polis 2008
Focus on BasicsFocus on Basics
http://www.ncsall.net/?id=31
Best practices – professional wisdom
6464McLendon & Polis 2008McLendon & Polis 2008
Current research on adult learning and literacy
How research is used by adult basic education teachers, counselors, program administrators, and policy makers
6565McLendon & Polis 2008McLendon & Polis 2008
Focus on Basics TitlesFocus on Basics Titles
ESOLResearchCorrections EducationModes of DeliveryWorkplace EducationYouthTransitions
TechnologyMathematics InstructionLearning From ResearchWriting InstructionStandards-Based EducationAccountabilityAdult Multiple Intelligences
6666McLendon & Polis 2008McLendon & Polis 2008
Curriculum Development"-isms“CounselingStaff DevelopmentLiteracy and HealthAdult DevelopmentFirst-Level LearnersResearch to Practice
Project-Based LearningTheories of ChangeThe GEDLearner MotivationContent-Based InstructionMulti-level ClassroomsReadingWhat is Research?
12
ERICERIC
Education Resources Information Center
http://www.eric.ed.gov/ERICWebPortal/Home portal
6767McLendon & Polis 2008McLendon & Polis 2008
Portal/Home.portal
Access to more than 1.2 million bibliographic records of journal articles and other education-related materials
6868McLendon & Polis 2008McLendon & Polis 2008
Putting the Four Steps into PracticePutting the Four Steps into Practice
One of the best sources for ACCESSING research
National Center for the Study of Adult Learning and Lit (NCSALL)
6969McLendon & Polis 2008McLendon & Polis 2008
Literacy (NCSALL)http://www.ncsall.net/
7070McLendon & Polis 2008McLendon & Polis 2008
7171McLendon & Polis 2008McLendon & Polis 2008
Best Resource for Program ManagersBest Resource for Program Managers
Program Administrators’ Sourcebook: A Resource on
7272McLendon & Polis 2008McLendon & Polis 2008
A Resource on NCSALL’s Research for Adult Education Program Administrators
13
Thoughts
Is there evidence that the strategy worked for others?Has it worked in programs similar to ours?Does it fit with other things we are doing?
7373McLendon & Polis 2008McLendon & Polis 2008
doing?Will we be able to afford it?Can we find someone who is using it so we can talk to them?
Planning and implementing a pilot test
Engage as many of your staff as possible in planning the pilot test.
Use your best teachers.
7474McLendon & Polis 2008McLendon & Polis 2008
Earmark necessary funds.
Planning and implementing a pilot test
Use your pilots to:Test the strategy to make sure it works.Make recommendations of any policy/procedural changes you will have to make.
7575McLendon & Polis 2008McLendon & Polis 2008
Make recommendations of how much money it will take to go to scale.Develop the professional development other teachers will need to use the new strategy.Conduct the professional development.
Measuring impact
Document the impact the new strategy had toward solving the problem.Document your developed performance indicators.Document the evidence collected that
7676McLendon & Polis 2008McLendon & Polis 2008
Document the evidence collected that staff indicated they needed to be convinced it worked.
Tweaking and going to scale
With the pilot site staff, make any necessary fine tuning.
Set up the professional development, make the policy/procedural changes, secure the funds tell your boss
7777McLendon & Polis 2008McLendon & Polis 2008
secure the funds, tell your boss.
Go to scaleIn waves if necessary
Activity 5 -Benchmarking the Steps
Plan ahead.
Meet your deadlines.
Sample benchmark chart Pg. 6Pg. 6
7878McLendon & Polis 2008McLendon & Polis 2008
14
QUESTIONS?
7979McLendon & Polis 2008McLendon & Polis 2008
QUESTIONS?
Quality First
It is all about
Program Improvement
8080McLendon & Polis 2008McLendon & Polis 2008
Getting Better Every Year at
Serving Adult Learners
Always willing to help…
Lennox [email protected]
Kathi [email protected]
8181McLendon & Polis 2008McLendon & Polis 2008
Reminder—First Steps
Benchmarks
By September 15, email your PPF [email protected]
Page 7Page 7
Page 6Page 6
8282McLendon & Polis 2008McLendon & Polis 2008
8383McLendon & Polis 2008McLendon & Polis 2008