tif-leap training fall 2009 susan kirkendol senior assessment program analyst scott reynolds student...
TRANSCRIPT
TIF-LEAP Training Fall 2009
TIF-LEAP Training Fall 2009
Susan KirkendolSenior Assessment Program Analyst
Scott ReynoldsStudent Learning Objective Specialist
Susan KirkendolSenior Assessment Program Analyst
Scott ReynoldsStudent Learning Objective Specialist
Review
Summary
• TIF – Teacher Incentive Fund
• LEAP – Leadership for Educators’ Advanced Performance
• 20 schools were selected
• Last year 10 schools completed the Student Learning Objectives Process
• This year all 20 schools will participate
The Design
• Centerpiece of the plan: Student Learning Objectives (SLOs)
• Teachers select a learning content from SCOS (or other state approved standards) – Selection is based on data
• Design an objective around that content with goals for student growth, teaching strategies– Provide rationale for choices made
• Incentive payouts are contingent upon goal achievement and a growth component
TIF-LEAP Goals
Devise and test an incentive plan that will:• Assist in the recruitment and retention of
highly effective and certified staff to teach every child
• Aid in recruiting teachers to high-needs schools
• Include more than EOG/EOC teachers• Focus on student learning• Improve instructional practices• Develop a systemic, comprehensive
compensation plan
Strategic Plan 2010 Goals
Goal I: High Academic AchievementGoal II: Effective EducatorsGoal III: Adequate Resources and
FacilitiesGoal IV: Safe and Orderly SchoolsGoal V: Freedom and Flexibility with
AccountabilityGoal VI: World-Class ServiceGoal VII: Strong Parent and Community
Connections
Background of SLO Process – Denver Pilot
• Piloted a similar program in Denver with 16 schools (at all 3 levels) with similar demographics over four years.
• Used objectives to link students' achievement to teachers' efforts.
• Program provided teachers with a high degree of autonomy.
• Research of the process was conducted by our grant partner, CTAC, using various statistical methodologies and arrived at several findings:
Impact of Objectives & Student Achievement
• Excellent objectives were associated with high mean achievement, as measured by their standardized tests (ITBS and CSAP).
• Meeting two objectives made a positive difference in mean student achievement.
• Objective setting is a critical initial step in the planning and delivery of instruction.
Background Effect on Pilot Results
• Achievement rose as length of time in the pilot increased.
• Ability to meet objectives increased with length of time in the pilot.
CMS Implementation
• We are in Year 3 of the 5-year grant.• Steering Committee meets monthly.• Other groups meet as needed.• Changes reflect input from this year.• With the goal of becoming the comprehensive
compensation program, TIF-LEAP has absorbed funding from – Pay for Performance– STAR Teacher & Administrator– High School Challenge– Master Teacher & Administrator
Steering CommitteeSteering Committee
Budget &
FinanceSheila Shirley
Dennis Covington
Budget &
FinanceSheila Shirley
Dennis Covington
CommunicationsLaTarzja Henry
CommunicationsLaTarzja Henry
Curriculum &
Instruction Ann Clark
Curriculum &
Instruction Ann Clark
Human Resources
Maurice Ambler
Human Resources
Maurice Ambler
Information Technology
Susan Johnson
Information Technology
Susan Johnson
Achievement Zone
Curtis Carroll
Achievement Zone
Curtis Carroll
Compensation SpecialistJan Millard
Compensation SpecialistJan Millard
Senior AssessmentProgram AnalystSusan Kirkendol
Senior AssessmentProgram AnalystSusan Kirkendol
InstructionalSupport WorkingGroup
InstructionalSupport WorkingGroup
Finance/HRWorking GroupFinance/HR
Working Group
PrincipalsWorkingGroup
PrincipalsWorkingGroup
SLO DesignTeam
SLO DesignTeam
Student Learning ObjectiveSpecialist
Scott Reynolds
Student Learning ObjectiveSpecialist
Scott Reynolds
Accountability&
AssessmentChris Cobitz
Accountability&
AssessmentChris Cobitz
Executive DirectorSusan Norwood
Sr. Admin. SecretaryCarol Turner
TIF-LEAP Organization
Overview
• Teacher identifies learner centered problems by reviewing data and student work
• Teacher begins to write Student Learning Objectives (SLOs)
• Teacher writes or selects an assessment to pre-test students knowledge/skills on this content
• Administrator approves the pre-test• Teacher administers the pre-test• Teacher submits SLOs for approval including:
– SLO PDF file – Excel SLO Student Score Sheet using pre-test results to
predict a minimum expected score– Copy of assessment(s)– Appropriate set of approved standards (available on
Riverdeep)
Teacher Eligibility
• Every certified, lateral entry, Teach for America, and VIF teacher must participate
• Must earn At- or Above-Standard ratings on each function of the TPAI-R
• Teachers must be present in the classroom for at least 85% of the interval specified for the SLO
Requirements to Participate
• Every certified teacher much complete TWO SLOs
• The first SLO will be written for an entire class• The second SLO will be one of the following:
– A Targeted Group SLO focusing on a group of students. This could be another class if appropriate. If the same class is selected for two SLOs, different objectives from the SCOS must be used.
– A Team SLO where a group of teachers work together to design common objectives and a common assessment for students.
More on Teams
• Who can be a team?– Teams could be grade-level or content-specific
teams– Teams can be inter-disciplinary– Team make-up must have the approval of your
administrator
What must be the same with team SLOs?– Learning content– Assessment
More can be in common than these two components, such as the case of inclusion teachers, etc. but these two components are the minimum that must be the same.
Payout Structure for SLOs
Payouts will be based on:
– Successful completion of the SLO(s)
– Having at- or above-standard ratings on ALL FUNCTIONS of the TPAI-R
Payout amounts:
– Will be $1400 for each successful SLO completion
– All certified teachers are required to complete at least TWO SLOs
New Growth Component
• New this year is AN ADDITIONAL incentive based on a growth component
• Actual dollar amounts are under discussion
• Will not affect SLO payout
Who Are TESTED Teachers?
THOSE WHO TEACH:• Grades 4 to 8 Reading or Math• Grades 5 and 8 Science• Algebra I• Algebra II• Biology• Civics and Economics• English I• Geometry• Physical Science• US History
Growth for TESTED Teachers
• Teachers who have State tests (as listed on the previous slide) associated with their class will be eligible for an incentive based on growth in these test scores.
• Average growth for a teacher will be calculated based on the scores of all students in tested subjects (EOC/EOG) with a minimum requirement that 5 students must be able to have growth calculated.
• If this growth score is above average, and all conditions of eligibility are met, teachers will receive an incentive.
Growth for TESTED teachers
TESTED TEACHERSREQUIRED:
• SLO 1 – Class
• SLO 2 – Targeted Group or Team
AUTOMATIC ELIGIBILITY:
• Growth measure based on tested subjects taught to all students for that area – minimum of 5 students.
Options for NON-Tested Teachers
Teachers who do not have EOG or EOC tests associated with their classes have 3 possibilities:
1. Team up with a colleague or a group of colleagues who teach tested subjects and receive an incentive based on the growth of the students taught by the non-tested teacher. It must be possible for growth to be calculated for at least 5 students. Contact [email protected] if you need further information. This must have administrator approval.
2. If option #1 is not possible: Complete a 3rd SLO with a possible incentive of $1400 for this SLO.
3. Complete the 2 SLOs only and do not seek participation in a team growth measure with no additional incentive possibilities.
Options for NON-Tested Teachers
NON-TESTED TEACHERSREQUIRED:• SLO 1 – Class• SLO 2 – Targeted group or Team
OPTIONAL:• Growth measure based on team OR
SLO 3 – Class, Team, or Targeted Group (Administrator approval required for growth measure and Team SLOs.)
Deadlines to Remember
Two windows for SLO submission• FALL
– OCTOBER 2: Deadline for Fall draft SLO submission by TEACHERS
– OCTOBER 16: Deadline for Fall SLO approval completion by ADMINISTRATORS
• SPRING – FEBRUARY 12: Deadline for Spring draft SLO
submission by TEACHERS– FEBRUARY 26: Deadline for Spring SLO approval
completion by ADMINISTRATORS
Population
Which students have you identified to be involved in your SLO?
• Should begin by looking at student data to identify a learner-centered problem
• What are their strengths?• What are their areas of need?
PopulationRequirements
• CMS identification numbers and names for each student are required.
• Class SLO = students seen at one time.• Targeted group SLO = students seen at
one time targeted for a specific need.• Minimum number of students in a Class
SLO or Targeted Group SLO or Team = 5.
• Only students in attendance 85% of identified interval will count when determining the outcome of the SLO.
Population – Options to Consider
• Special cases may have population of less than 5:– Possibly EC, ESL, TD teachers, and where
pullouts are used– NOTE: Requires administrator approval
• If you want to add students who come after SLO start date – You MUST state how they would be included in your SLO before approval for implementation– NOTE: You will not be able to add them to
the Student Score Sheet until the end of the interval
– Students still must be present for 85% of the interval.
Population – Restrictions
• The same population cannot be used for both SLOs UNLESS the learning content is different.
• If no students meet the 85% attendance criteria, the teacher is not eligible for the incentive.
• Students not present for at least 85% of the interval will not count for or against the teacher for the SLO’s outcome.
• The population must be students seen during regular school hours.
Population – Options
• If you have a highly transient population:– Consider buffering the number in the group to protect
against falling below the minimum of 5 students.
• If you have flexible grouping:– Plan for your students to be in the group for at least
as long as your interval (43 days minimum) regardless of the growth achieved.
– Consider shorter intervals where you are more likely to maintain the same student roster.
Population Subcomponents
• How many students are included in your population? (Will be identified with names and ID numbers on the SLO Student Score Sheet.)
• Assigned grade level(s) of students in the population.
• How often do you see these students within the interval?
• What percentage of students much achieve their growth expectation if less than 5 students are present at least 85% of your interval?
• Describe the data reviewed for this selection and how it led to these students.
• What are these students’ strengths?• What are these students’ areas of need?
What is the SLO Student Score Sheet?
• This is an EXCEL spreadsheet• You will enter student’s names, ID numbers,
pre-test date and score, and other requested items
• Each of the columns in white must be completed prior to SLO implementation
• Student Score Sheets MUST be submitted with the SLO
• Post-test scores and growth scores will be added to this same sheet once the SLO is completed
Improving readership
Data Resources
Student Achievement, Attendance, Demographic Data
• T-sparta (by student)• I-sparta (by school/grade)• NC Wise• Cumulative Folders (report cards, IEPs)• SAS EVAAS• Goal Summary reports• Common formatives• Data inventory
Improving readership
Data Resources
Other sources
• School Improvement Plans – – What are the goals for your school?
• Parents/Families
• Students themselves– What was easy? What was hard?
Improving readership
Data Resources
Student Performance Data
• Writing samples
• Projects
• Homework
• Presentational pieces – art, music, PE, oral reports
• Student comments
Improving readership
Data Resources
Teacher-Generated Data
• Interest surveys
• Learning styles questionnaires
• Multiple intelligences surveys
• Pre-testing
Learning Content
The selection of objectives you will teach from one of the following:
– NCSCOS– CTE: Blueprints– ESL: Grade-specific ELA objectives – PreK: Approved Widely Held Expectations from
Foundations Standards– AVID: Approved Objectives– JROTC: Approved Objectives– Other: Contact [email protected]. (For
grades 6-12, only courses in the 2009-2010 Planning Guides can be used. If this presents a problem, email Scott.)
Learning Content Requirements
• At least 2 objectives must be chosen from the set of standards for each SLO.
• Math: Each lettered objective counts as a single objective.
• ELA, Science, and others: Bulleted items do NOT count as a single objective.
• If an objective has multiple bullets – at least ONE bullet must be selected.
• Teacher must be certified to teach in the content area selected. – (Team SLOs may be an exception to this rule with
administrator approval.)
Learning Content Restrictions
• When 2 SLOs are written . . . – Learning content for the SLOs cannot be the same
UNLESS the population is different for the 2 SLOs
THIS IS PERMISSIBLE:SLO 1 SLO 2
Learning content X Learning content X
Population A Population B
THIS IS NOT ALLOWED:SLO 1 SLO 2
Learning content X Learning content X
Population A Population A
Learning Content Restrictions
• Reading comprehension is too global to select for learning content.– Instead target specific aspects of reading
comprehension such as:• Character analysis• Plot development• Drawing conclusions• Interpreting figurative language
• K-12 Skill Competency Goals and Objectives for Social Studies are too global as well.– These may only be used if at least 2 grade-
specific objectives are also selected.
Learning Content Restrictions
• ESL teachers will use English Language Arts objectives for the grade/course represented.
Learning Content – Cautions
• Learning content should align with the interval chosen.– Consult your pacing guide. – Make sure the content will require at least a
quarter to be learned.
Learning ContentSubcomponents
• Content area for the SLO• Selected objectives from the approved
set of standards• Describe the data reviewed for this
selection and how it led to these objectives. Include any needs assessments and/or baseline data sources used.
• What evidence can you provide that your content is rigorous?
Interval
Interval is the period of time during which the SLO will be completed.
• Must state a start and stop date.• Teachers must be in attendance for at
least 85% of the interval selected.
IntervalRestrictions
• The shortest interval allowed is the number of days in the shortest instructional quarter for the year.– This year this is 43 days
• The longest interval allowed is the school year.
IntervalRestrictions
• Intervals cannot change once they are approved.
• Make sure you allot enough time for students to learn the content.
• Have plans about how to handle the process if students learn the material faster than expected.– Make sure these plans are clearly stated
in your SLO.
IntervalSubcomponents
• Interval start and stop dates• Number of instructional days in
interval• Middle and high school teachers will
enter:– NCWISE title of the course– NCWISE numeric course code– During which period will you teach this
SLO?
• During which quarter(s) is your interval?
IntervalSubcomponents
• Why did you select this interval?• How does your interval align with
local pacing for the content?• If your students fully learn the content
before the end of the interval, what is your plan?
• How will you ensure content depth and complexity within the interval?
Assessment(s)
• What instrument(s) will you use to measure student learning of the content?– Assessments can be
• Existing assessments• Developed with a team (i.e., common
formatives)• Teacher-designed
Assessment(s) Requirements
• Each SLO must have at least one pre-assessment and a post-assessment measure.
• Each objective of the learning content must be measured by at least one item on both the pre- and the post-assessment.
• All assessments must be approved by an administrator before they are given to students.
Assessment(s) Requirements
• Assessments must be submitted in electronic format along with your SLO.
• The pre-assessment(s) must be administered before your SLO can be approved.– Scores are needed to make predictions
about growth expectations.
Assessment(s) Requirements
• If divergent items (more than one correct answer possible) are used - a scoring guide or rubric must be developed and attached in electronic format.– A scoring guide specifies for each item how
much credit will be given for correct and partially correct responses.
– A rubric is used to measure tiers of student performance for a comprehensive task.
Assessment(s) Options
• Using more than one assessment may result in a more reliable measurement of what students have learned.
• Use both divergent (more than one correct answer possible) and convergent (a single correct answer) to measure what students know more accurately.
• You may use the same test for the pre- and the post-assessment.
Assessment(s) Options
• Rubrics can be shared with students in advance with administrator approval. – Scoring guides are similar to answer
keys and cannot be shared with students in advance.
• You can allow partial credit for answers but make sure the scoring guide clearly describes how this would work for each item.
• You may give re-tests if given administrator approval at the time of SLO approval.
Assessment(s) Restrictions
• The pre- and post-assessments cannot be used as instructional materials in the class
• Assessments, rubrics, and scoring guides cannot be changed once they have administrator approval
Assessment(s) Cautions
• Even if developed by an “expert” – every test item may not be a good item.
• Balance objectives across the assessment according to you how you wish to weight them.
• Many assessments, including “ready-made” assessments or those created from testing software may have a lack of quality alignment and/or limited reports. – Make sure you will be able to collect the
data you will need to evaluate your SLO.
• Count only those items that are related to the learning objectives chosen.
Guidelines for Assessments
Consider each of the following:
• Alignment with SCOS or other appropriate objectives
• Bias Elimination
• Reliability
• Validity
• Growth
Formative vs. Summative Assessments:
• Formative Assessment
* is a PROCESS * is assessment FOR learning * conducted to improve and promote student
success
* conducted during learning
• Summative Assessment
* is a PRODUCT
* is assessment OF learning * conducted to grade student success
* conducted at the end of learning
Assessment Basics
Convergent vs. Divergent
• Convergent– Items where there is one
correct response
– Typically use lower order thinking
– Types:• Multiple choice• True/False• Matching• Fill in the blank
• Divergent– Items where there may
be more than one correct response
– Typically use higher order thinking
– Types:• Essays/papers• Portfolios• Projects• Performance
evaluations
Assessment Basics
Assessments Recommendations
• There should be at least 2 items to measure each objective
• Use multiple forms of measure including some convergent and some divergent items
• Tests should be rigorous– Rigorous implies depth of assessment
in terms of content and thinking skills required
– Rigorous does NOT mean length
Assessments Subcomponents
• For both the pre- and the post-tests:– Total number of items– Total number of possible points– How many assessments will you use?– Are the assessment items convergent
(one correct answer), divergent (more than one correct answer), or a both?
– Names of the tests
Assessments Subcomponents
• Are you using the same, exact assessments pre- and post?
• How many re-tests of the post-assessments are allowed?
• Is there a scoring guide and/or rubric for each divergent assessment?
Assessments Subcomponents
• Have any/all assessment items been used before? If so, explain.
• Where and from whom did the assessment(s) originate?
• Describe how you have ensured the assessment(s) is/are aligned to the content.
• Why is this the best way to assess student learning of your selected learning content?
• How free of bias is/are the assessment(s) (such as vocabulary; scenarios, etc.?
Assessments Subcomponents
• Describe any testing modifications you will use and whether a proctor will be present.
• If using multiple assessments, how will a single pre-test score and single post-test score be calculated?
Growth Expectations
What are your expectations for how students will grow in their learning of the selected content over the interval, as measured by the pre- and post-assessment(s)?
Growth ExpectationsRequirements
• Each student in the population must have a growth expectation that is clear.
• At least 75% of the students included in the SLO must achieve the growth expectation in order to receive the compensation. (Higher if teacher so states.)
• Expectations must be rigorous and attainable and agreed upon between the teacher and the administrator.
Growth ExpectationsRequirements
• The growth expectation for each student will be shown on the Student Score Sheet in the “minimum expected post-test score” column.
Growth ExpectationsOptions
• Growth expectations can be differentiated in many ways, including– Setting different growth expectations for the top third,
middle third, and bottom third of students or– Setting different growth expectations for each
student
• You may contact TIF-LEAP or Curriculum and Instruction for assistance in selecting appropriate growth goals.
• You may indicate that more than 75% of your students will achieve expected growth, but that is the standard to which you will be held.
Growth ExpectationsCalculation
• You may use any method for calculating growth that works for you BUT:
• Remember that on the SLO Student Score Sheet you will be asked to list an expected score for EACH student.
SLO Student Score Sheet
On the SLO SSS you will record the following items when you create your SLO:
1. RAW PRE-TEST SCORE – NUMBER OF POINTS EARNED (no percents)
2. The number of possible points on the pre-test
3. Minimum expected score on the POST-test. This is the score you want your students to reach at the end of the SLO interval
DO NOT REPORT PERCENTS, INCLUDE PERCENT SIGNS, or FRACTIONS
SLO Student Score Sheet
Once your SLO has been completed,you will return to the SLO SSS and
complete the following columns1. RAW POST-TEST SCORE – Number
of points earned --- NO PERCENTS2. The number of possible points on the
POST-TEST3. Growth made = RAW POST-TEST
score MINUS RAW PRE-TEST score4. A yes or no column indicating whether
the student met the goal set or notDO NOT REPORT PERCENTS, INCLUDE
PERCENT SIGNS, or FRACTIONS
Growth ExpectationsCalculation Options
• How many more words per minute will be read? – My students will increase their reading by at least 25 words
per minute. (Here you would enter an expected score – so a student scoring 25 on the pre-test should have a 50 recorded as minimum expected post-test score)
• How much growth will students make across one or more parts of a rubric?– I want to see all of my students move up at least 1 level on
all criteria on my rubric. So, a student who scores in the “needs revision” level should move up to “competent”, those in the “competent” level should move up to “good”, those in “good” should move up to “outstanding”. Those in “outstanding” range already should maintain that status on all of the criteria. (We would prefer to see these designated as numbers if at all possible).
Growth ExpectationsRestrictions
• Vague statements of growth cannot be accepted.
• Make sure there will be no question about whether a student made the expected growth or not. Specify a clear number in the minimum expected score column as a goal for each student.
Growth ExpectationsCautions
• Plan ahead for potential loss of students. – If you want 85% of your students to meet your growth
expectation and you begin with a class of 20, you would need 17 to meet the goal. But if you reduce to only 10 students who are present the required time, you need 9 of 10 to meet the goal – obviously a little tougher to accomplish.
• Students who score high on a pre-assessment have very little room to grow. – You may want to state that they will maintain their
score, or within a narrow range of their score.
Growth ExpectationsCautions
• Be sure you will be able to determine if this type of score can be obtained from the test chosen.– Some tests do not allow you to determine scores by
goal or objective – they just result in an overall score. These are not suitable for use with SLOs.
• When developing rubrics and/or scoring guides, be sure to consider all types of possible answers they could give.
Growth ExpectationsSubcomponents
• What percent of students must achieve their minimum expected score?
• Describe the data reviewed and how it led to these growth expectations.
• How were growth expectations calculated for each student?
• How do you know your expectations are rigorous and attainable?
StrategiesRequirements
What instructional techniques and approaches will be used to facilitate student learning?
Requirements:• List multiple strategies for each SLO.• Each SLO must clearly align with
– The School Improvement Plan– The teacher’s Individual Growth Plan
StrategiesOptions
• Using effective, research-based instructional strategies is recommended.– Innovative strategies may be used with
administrative approval.
• Work with your administrators to ensure all subjects are covered in the SIP.
StrategiesOptions
Analysis of student workConcept/Thinking MapsConstructivismCooperative LearningDirect InstructionEffective QuestioningExploring MisconceptionsEntrance/Exit TicketsField Experiences/TripsFlexible/Strategic GroupingFoldables®Formative AssessmentsGallery CrawlGraphic OrganizersGuest SpeakersGuided PracticeHands-On InstructionHomework Independent PracticeInquiry
Inside-Outside CircleIntegration of Content AreasInternational ConnectionsInterviewsJigsawJournaling/NotebookingLaboratoriesLearning CentersLearning Styles FocusLeveled Reader UseManipulativesModeling (Teacher or Student)MnemonicsMultiple Intelligence Focus Note-Taking Paideia/Socratic SeminarsProjectsQuickwritingReader’s Theatre
Reinforcing and RecognizingShared/Paired PracticeSimulation/Role-PlayStudent Peer-Assessment Student Self-AssessmentTechnology IntegrationThink-Pair-ShareTotal Physical ResponseVisual CuesWhole Language Word Wall Other:
StrategiesCautions
• Increased student engagement does not necessarily result in improved achievement. (Compliance is different than engagement.)
StrategiesSubcomponents
• You will select strategies you intend to use in this SLO from the list
• Connection to the School Improvement Plan:– Aligned SIP outcome– Aligned SIP strategy– Aligned SIP task
• What other connections does this SLO have to your SIP or other initiatives to be implemented?
• What IGP goals, strategies, etc., are addressed?
StrategiesSubcomponents
• How will you differentiate instruction?• Please provide a brief statement as to
what research and/or other data supports the selected strategies.
• Which, if any, of the selected strategies are new for you to implement?
• What support and/or professional development will you see/request to help achieve your SLO?
StrategiesRestrictions
• Instructional strategies should be age, grade, ability level, and content appropriate.
Questions?
Remember to reference the Implementation Guide for SLOs found on Riverdeep for assistance.
Contact Information
General TIF-LEAP Contact Information
980-343-1124
Susan Kirkendol
980-343-1609
Scott Reynolds
980-343-2708