trail blazers
DESCRIPTION
Trail Blazers. Navigating the Landscape of the Smarter Balanced Field Test with ALL Students . June 25, 2014 10:15 – 11:45 am NCSA – Next Generation Assessment Systems . Trail Blazers. Idaho - Angela Hemingway Montana – Judy Snow South Dakota – Jan Martin . Session Overview . - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/1.jpg)
Trail BlazersNavigating the Landscape of the Smarter
Balanced Field Test with ALL Students
June 25, 201410:15 – 11:45 am
NCSA – Next Generation Assessment Systems
![Page 2: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/2.jpg)
Trail Blazers
• Idaho - Angela Hemingway• Montana – Judy Snow • South Dakota – Jan Martin
![Page 3: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/3.jpg)
Session Overview
• Each state will describe the journey with details addressing such topics as:– Decisions as to why all students in the field test– Balancing new assessments with existing
systems – Political landscape – Communication strategies across all stakeholdersThere will be time for questions and discussion at the end of the session.
![Page 4: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/4.jpg)
Idaho’s JourneyAngela Hemingway
Director, Assessment and Accountability
![Page 5: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/5.jpg)
Brief Overview of Statewide Assessments in Idaho
1998 2000 2002 2004 2006 2008 2010 2012 2014 2016
Idaho adopts standards in core subjects
First version of
ISAT (NWEA)
Second version of ISAT (DRC)
Direct Writing &
Direct Math End
Idaho adopts higher
standards in Math & ELA
Idaho joins
Smarter
Smarter Pilot Test
Idaho Core Standards
Implemented & Smarter Field Test
Smarter OperationalTest, Digital Library, & Interim
Assessments
Idaho revises
standards in core subject areas
Online statewide testing begins
![Page 6: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/6.jpg)
Why test ALL students? • New standards were to be implemented SY13-14• Some push back from teachers who were concerned
that accountability was tied to the “old tests” and “old standards”
• Some were not going to teach the new standards if they were going to be held accountable to the old standards
• US DOE flexibility allowed for a chance to avoid double-testing
• Chance to test our infrastructure• Allow for students, teachers and administrators to
experience a “dress rehearsal” before the 2015 operational assessments
![Page 7: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/7.jpg)
Testing All Students – Logistics and Technologies
• All students in Grade 3-8 and 11 took both the math and ELA non-PT and PT
• Over 164,000 students participated with an 88% completion rate
• Grade 9 and 10 became optional due to test length (~70% of districts chose to test 9 and 10)
![Page 8: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/8.jpg)
What were the students’ perceptions?
• Nearly 10,000 students participated in the statewide survey
• Surveys were developed by grade band • Paper and electronic versions were made
available to all schools
![Page 9: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/9.jpg)
pretty hard
somewhat hard
somewhat easy
pretty easy
0 200 400 600 800 1000 1200 1400 1600 1800 2000
3-5 6-8
9-11
What did you think about the questions on the test you took today?
![Page 10: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/10.jpg)
pretty hard
somewhat hard
somewhat easy
pretty easy
0200
400600
8001000
12001400
16001800
2000
3-5 6-8
9-11
What did you think about navigating the test you took today?
![Page 11: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/11.jpg)
not very well
somwewhat well
very well
0 500 1000 1500 2000 2500 3000
3-56-89-11
How well did the test you took today match what you learned in class this year?
![Page 12: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/12.jpg)
85%
13% 2%
Yes, I knew how to use the keyboard to type my answers.
Yes, but it was hard to use the keyboard to type my answers.
No, I did not know how to use the keyboard.
Grade 3- 5: Were you able to use the keyboard to type your answers?
![Page 13: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/13.jpg)
Students Comments –LIKES
![Page 14: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/14.jpg)
Students Comments – DISLIKES
![Page 15: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/15.jpg)
Technology Coordinator Survey - What went well with the administration of the
Smarter Balanced Field Test in your district?
![Page 16: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/16.jpg)
Technology Coordinator Survey - What were the biggest challenges your district faced in administering the
Smarter Balanced Field Test?
![Page 17: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/17.jpg)
Technology Coordinator Survey - How did supporting the administration of the Smarter Balanced Field Test in your district
compare to supporting administration of the ISAT in previous years?
![Page 18: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/18.jpg)
Principal Survey - What new expenses do you anticipate to incur to administer
Smarter Balanced next year?
![Page 19: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/19.jpg)
Principal Survey - What went well with the administration of the Smarter Balanced Field
Test in your district or school?
![Page 20: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/20.jpg)
Principal Survey - What were the biggest challenges your district or school faced in
administering the Smarter Balanced Field Test?
![Page 21: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/21.jpg)
Idaho’s Changing Political Landscape• Significant support early in the process• All looking forward to higher standards and more
robust assessments• Then it became publicized that no ISRs would be
available• “Opt outs” began….but do not appear to be
significant in number (survey in progress)• Many schools are changing their policies regarding
enrollment and attendance to avoid this situation in the future
![Page 22: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/22.jpg)
What we have learned?
• Information needs to be thoroughly reviewed internally before its public release
• Training modules and documents are helpful, but need to be streamlined
• Formation of a Smarter Balanced Committee helped determine areas of additional need/support and should be formed sooner
• Edmodo site was very helpful, but required significant staff time; guidance should be provided regarding appropriate posts
• Surveys were helpful, but need slight modifications
![Page 23: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/23.jpg)
Montana’s Journey
![Page 24: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/24.jpg)
• 56 counties• 316 districts• 824 schools• 332 schools with under 50
students• 71 one room school houses• Big Sky Country
253 Miles: Longest Bus Route
![Page 25: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/25.jpg)
Why Field Test All Students
• Commitment to CCSS• Support for Computer Adaptive testing• Partnerships• Dress rehearsal/ trial run/pressure test
system
![Page 26: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/26.jpg)
The New and the Old
![Page 27: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/27.jpg)
Political Landscape
• Paper pencil tests• Misinformation• Opt out• Myths
![Page 28: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/28.jpg)
Technology
• Number of Montana students starting and completing tests: 72,876 (98%)
• Only one school without internet capacity
• Technology Assistance Initiative– Contract with META and SAM
• Montana Educational Technologists Association
• School Administrators of Montana• Regional meetings• Final report
![Page 29: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/29.jpg)
Communication
• July 2013—Meetings with partners• September 2013—official email request waiver• Dedicated newsletters and website• January Assessment Conference• Live weekly
– 20 minute digital blasts (recorded)– Webinars (recorded)
• Transition liaisons• Crash course• Surveys• Montana educator involvement in development
activities and Digital Library
![Page 30: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/30.jpg)
Advice for Districts and Schools
• Form a Team• Use Tech Readiness Tool• Use Practice Tests• Pilot and debrief
![Page 31: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/31.jpg)
Survey CommentsThe test went very smoothly. The students were engaged and took the challenge (and it was very challenging) seriously. Everything worked well at our school because of good planning, organization and communication.
Navigation was too complicated for many of our students.
Special ed students were not able to access their accommodations.
I thought that there were very good questions that challenged the students and made them think about the mathematics.
Went quite smoothly - no problems at all.
There were some glitches in the test that kept the students from answering some problems with certain answers. For example
Students feedback was that they enjoyed being able to scroll and check the questions before starting the reading passages. Flagging questions and the glossary were also mentioned as tools they enjoyed
![Page 32: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/32.jpg)
Next Steps
• Accommodations• Practice Tests• MOU and Vendor Contract• Statewide Training Plan
![Page 33: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/33.jpg)
South Dakota’s Journey
![Page 34: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/34.jpg)
Starting with a decision
• Why test students on old standards?• US DOE flexibility allowed for a chance to
avoid double-testing.• Need to pressure-test the technology
infrastructure in the state.• Allow for students, teachers and
administrators to experience the test environment before it counts.
![Page 35: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/35.jpg)
Navigating the uncharted waters of a double-testing waiver
• Timing of decisions and information from US DOE.
• Getting specific commitments from districts with Hutterite Colony students.
• What to do with students in JDC or other placements?
• Worked with current vendor to provide needed paper/pencil tests to ensure all students would be tested based on waiver expectations.
![Page 36: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/36.jpg)
Smarter Balanced in South Dakota by the numbers
• 150/151 Public school systems• 20/20 BIE/Tribal systems• 36/53 Non-public systems
• 70,560 student took tests • 279,137 tests started • 278,164 tests submitted• 1,200+ district personnel involved• 4 SD DOE Staff with tech support from BIT
![Page 37: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/37.jpg)
Were the schools ready?
• Yes and no. • A key learning was while the schools have
the technology, how it is used with students vary across the state.– Schools with one-to-one Chromebooks,
iPads, or other devices – less student issues.
- Schools with limited access especially in the elementary, more significant issues.
![Page 38: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/38.jpg)
Were the schools ready?
![Page 39: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/39.jpg)
Balancing new assessment with existing systems
• Science still paper and pencil - regular and alt• Alternate assessment included in the waiver with field test
participation voluntary at school level• ELL testing still paper and pencil for another two years • Schools needing paper/pencil reading and math tests• Tech readiness • Changes in procedures and timelines• School staff overwhelmed with all the changes
![Page 40: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/40.jpg)
Balancing new assessment with existing systems
• Accommodations was the number one issue for most schools
• Calculators • Read aloud versus text to speech• Accommodations versus designated
supports• Resources to support the schools http://
doe.sd.gov/octe/SMARTERbalanced.aspx
![Page 41: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/41.jpg)
Special Education
• Fall workshops across the state on shifts in accommodations for on-line testing
• Spring testing workshops• Supports provided by SBAC and SD DOE
• The shift to on-line testing and changes in accommodations (including language used) was one of the biggest hills we still are climbing to assist educators in understanding.
![Page 42: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/42.jpg)
Political Landscape
• 7 bills and 2 resolutions addressing CCSS and assessments
• 2 bills would have restricted ability of SD DOE to even work with out-of-state entities
![Page 43: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/43.jpg)
Breaking news…. Dateline June 22, 2014
• Oddly, the Republican convention chose not to ride another big conservative hobbyhorse, Common Core paranoia. Offered an opportunity to debate the oft-bashed curriculum standards, the convention instead approved a resolution that doesn't mention Common Core but requires that any multi-state standards and tests be approved by the Legislature... because of course, Republicans like to involve big government in education, as long as it's a government that they control
![Page 44: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/44.jpg)
Communication strategies for various stakeholders
• Administrators – weekly updates through Secretary of Education, updates on department website including webcasts, area administrator meetings
• Tech coordinators – weekly updates, area tech meetings, direct one-on-one support as needed
• Test coordinators –weekly updates from
Assessment Office, updates on department website including webcasts , webinars, workshops
![Page 45: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/45.jpg)
Communication strategies for various stakeholders
• Teachers – monthly updates from on-line newsletters , updates on department website including webcasts
• Parents – news media, letters/newsletters sent by schools, district websites
• Legislators - face to face meetings, presentations at committee hearings, one-on-one discussions
• General public – media, South Dakotans against the
common core, State Board of Education meetings
![Page 46: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/46.jpg)
Survey says….
![Page 47: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/47.jpg)
Themes that emerged from the survey feedback:
• If students and staff did practice tests, the field test went smoother
• Tech staff must be involved • Shift to on-line testing created confusion
and concerns with accommodations• Text-to-speech and sound the biggest
problem across all respondents.• Help desk was used with varying success
![Page 48: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/48.jpg)
Themes continued
• Training materials not necessarily sufficient or timely
• Too many last minute changes in procedures• Wide variance in district level communication
procedures• Typing/keyboarding an issue for younger
students • Misconceptions about Smarter Balanced,
field tests, and no scores
![Page 49: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/49.jpg)
Mapping the Future
• More targeted professional development to specific groups, both in-person and web-based
• Direct meetings with tech coordinators• Manuals and training materials must be teacher-
friendly• Small districts – need more staff to support the
district test coordinator (typically the high school counselor)
• Communication on blueprints, achievement standards, and claims – reporting results
![Page 50: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/50.jpg)
Some comments from the field
• Overall, for the field test, it went well. We all knew the test would be challenging. As a parent, my own son came home and shared that he was very glad that the writing part could be done with a keyboard. He state, “I don’t like to write, but if I could do it on a computer,…that makes it a lot easier! I wrote a lot for the Smarter Balanced test!”
![Page 51: Trail Blazers](https://reader035.vdocument.in/reader035/viewer/2022062520/56815ee7550346895dcd9484/html5/thumbnails/51.jpg)
And on the other hand…
• None (when asked about any positives to share). It is just another reason why students are hating school. Smarter Balanced is another private corporation going to the bank with public school money.