welcome!

Office of Standards, Assessments and Research & Evaluation

Welcome!

Assessment Subcommittee Meeting

July 17, 20109 AM – 4 PM

Colorado State University, Pueblo


AgendaWelcome, introductions and meeting goalsBackground InformationAssessment System revision process and roles11:30 – 12:30 LUNCHSubcommittee Breakout MeetingsLarge Group Debrief Adjourn


Meeting Goals Provide background information which lead to the

revision of the Colorado Assessment System. Understand the structure of the revision process

including goals and timelines. Identify the key players involved in the

assessment system revision and their roles and responsibilities.

Establish subcommittee norms, goals and expectations.


SB08-212 – KEY POINTS

Postsecondary and Workforce Readiness

School Readiness

Descriptions of “School Readiness” and “Postsecondary and Workforce Readiness”

Revised Standards (Fewer, Clearer, Higher)

Assessments and Endorsed Diplomas(High School Pilot)

Higher Ed Admission Standards


Fewer: Essential Concepts & Skills vs. Excessive Content

Clearer: Coherent Focus on the Student Outcomes

Higher: Greater Depth of Knowledge and Skills

What Is Fewer, Clearer, Higher?


Content Area: Mathematics

Standard: 2. Patterns, Functions, and Algebraic Structures

Prepared Graduates: Understand that equivalence is a foundation of mathematics represented in numbers, shapes, measures, expressions, and equations

Grade Level Expectation: Eighth Grade

Concepts and skills students master:

2. Properties of algebra, equality, and inequality are used to solve linear equations and inequalities

Evidence Outcomes 21st Century Skills and Readiness Competencies

Students can:a. Use the distributive, associative, and commutative

properties to simplify algebraic expressions b. Solve one-variable equations including those involving

multiple steps, rational numbers, variables on both sides, and the distributive property

c. Solve inequalities in one variable including negative coefficients and graph the solution on a number line

d. Represent the distributive property in a variety of ways including numerically, geometrically, and algebraically

Inquiry Questions: 1. Do algebraic properties work with numbers or just

symbols? Why?2. Why are there different ways to solve equations? 3. How are properties applied in other fields of study?

Relevance and Application:1. Procedural fluency with algebraic methods allows use of

linear equations and inequalities to solve problems in fields such as banking, engineering, and insurance. For example, it helps to calculate the total value of assets or find the acceleration of an object moving at a linearly increasing speed.

2. Comprehension of the structure of equations allows use spreadsheets effectively to solve problems that matter such as showing how long to pay off debt, or representing data collected from science experiments.

Nature of Mathematics:1. Mathematicians can think both forward and backward

through a problem. An equation is like the end of a story about what happened to a variable. By reading the story backward, and undoing each step, mathematicians can find the value of the variable.


21st Century SkillsColorado's description of 21st century skills is a synthesis of the essentialabilities students must apply in our fast changing world. These essentialsskills are:

Critical thinking and reasoning (for example, but not limited to: problem solving, analysis, logic, cause/effect)

Information literacy (for example, but not limited to: knowledge acquisition, source discernment, systems management)

Collaboration (for example, but not limited to: synergy, team resourcing, social skills, leadership)

Self-direction (for example, but not limited to: adaptability, initiative, personal responsibility, work ethics, self-advocacy)

Invention (for example, but not limited to: creativity, innovation, integration of ideas)


Transition to the CSAP Replacement

• Old Standards: Old Colorado Model Content Standards• Old Assessment: CSAP

2010-2011-

• New Standards: Colorado Academic Standards• Modified Assessment: CSAP assessment items which

align to the New Colorado Academic Standards only

2011-2012-

• New Standards: Colorado Academic Standards• Modified Assessment again: CSAP assessment items

which align to the Colorado Academic Standards and new assessment items

2012-2013-

• New Standards: Colorado Academic Standards• New Assessment: New assessment items

2013-2014-

• Old Standards: Old Colorado Model Content Standards• Old Assessment: CSAP

2010-2011-

• New Standards: Colorado Academic Standards• Modified Assessment: CSAP assessment items which

align to the New Colorado Academic Standards only

2011-2012-

• New Standards: Colorado Academic Standards• Modified Assessment again: CSAP assessment items

which align to the Colorado Academic Standards and new assessment items

2012-2013-

2013-2014-


Mantra Signals for mastery and readiness Easy to use Meaningful Timely


Assessment System Redesign

A redesign of Colorado’s assessment system must address these components:

ESEA (summative, grade-level and content specific) Special populations (special education, ELL, Gifted) School Readiness New Academic Standards Postsecondary and Workforce Readiness CBLA – Colorado Basic Literacy Act


Process and Roles Your role and responsibilities Stakeholders – role, structure,

meeting format Subcommittees – role, structure and

meeting format Other Key Players and Support


Roles and Responsibilities Stakeholders

Advisory role to CDE in the design of the assessment system revision process

Develop initial over-arching recommendations for subcommittee to utilize when discussing specific assessment options

Serve as a resource when necessary in subcommittee work

Analyze subcommittee product and complete final draft recommendation with considerations of financial and technical viability


FOUR GUIDING PRINCIPLES FOR COLORADO’S ASSESSMENT REDESIGN

(as agreed upon by the CDE Assessment Stakeholders Committee, January 11th, 2010)

1. A primary goal of Colorado’s PK-12 Assessment System is to foster the preparation of all students’ postsecondary and workforce readiness (PWR).

2. Colorado’s PK-12 Assessment System is a balanced system combining formative and summative components that measure both growth and status.

3. Colorado’s P-12 Assessment System is comprised of components that reflect the belief that the opportunity to demonstrate success is a right for all Colorado students.

4. Colorado’s P-12 System must be relevant for students and teachers in that it is tied directly to students’ learning and preparation for PWR, and it provides data/information on growth and status that informs instruction and is easily understandable by students, teachers, and parents.


Stakeholder Meeting PlansMeeting 1 – July 9, Colorado Springs, Antlers Hilton“Work Expectations”

Meeting 2 – August 6, Denver DTC Doubletree“Policy Plan”

Meeting 3 – September 20, Fort Collins Hilton“Analyzing subcommittee work”Guest Presentation – RFP writing/price sheet vendor – reality check Meeting 4 – September 24, conference callReceive TAC feedback and answers and resolve any issues before public

feedback tour begins Meeting 5 – October 15, Golden, Denver Marriot West “On-Time Delivery of Final Product”


Subcommittee Structure CDE received more than 400 applications from highly

qualified candidates

Over 90 members represent the following sectors: PreK, Elementary, Middle, High School, Higher Education, Business, Military, Parents; Geographic Region; Experience with Gifted and Talented, Students with Disabilities and ELLs; and, Experiences with Assessments.

All subcommittees will include some standards revision subcommittee members.

Subcommittee co-chairs

Support from CDE – each subcommittee will have two facilitators to serve as resources


Assessment Subcommittees Summative (son of CSAP) Postsecondary and Workforce

Readiness School Readiness Special Populations Formative and Interim


Subcommittee QuestionsSummative

What does it measure? What does it not measure? Should it measure growth? Are content and skills measured immediately or at the end of

year? What grades should be included – just 3-8 or also 1 and 2? How does it fit with ICAP? Is it exclusively online and what would that look like? Is it computer adaptive? Will it include a project or performance event? What is the maximum length of time acceptable between

assessment and results delivery? How will it be actionable and relevant to all stakeholders?


Subcommittee QuestionsPostsecondary and Workforce Readiness

What does it measure? What does it not measure? Should it measure growth? Is it a placement test? Is it an admissions test? Is it linked to the Colorado Growth Model? Is it linked to endorsement diplomas? Or graduation? Will it be through course or end of course? Is it linked to grade level or course advancement? What grade(s) will it assess? Is it content specific? Will it be a separate assessment or embedded in the summative? Will it demonstrate different levels of depths of knowledge? How does it reduce collegiate or corporate remediation? How will it be predictive over time after graduation?


Subcommittee QuestionsSchool Readiness

What does it measure? What does it not measure? Should it measure growth? Common minimum/shared indicators? Developmental and/or

content? Is it standardized? Is a student identifier number employed? Do they begin measurement of progression towards PWR? Should it immediately show a link to the PGCs? Does it show the strengths and weaknesses of a student? Should CBLA be re-written? Is PWR a grown up individual school readiness plan? Where does Kindergarten through 2nd grade live in the

Colorado Assessment System? And, which subcommittee should be responsible for the recommendation?


Subcommittee QuestionsSpecial Populations (1%, 2%, ELL, GT)

What does it measure? What does it not measure? Should it measure growth? Do we develop non required assessments for special populations? What should be considered in development of standardized

assessments that yield valid, reliable and comparable results for all students?

What is the purpose of assessments for special populations? What isn’t the purpose?

What assessment accommodations should be available for students? What alternative assessments are needed for School Readiness? What alternative assessments are needed for PWR? If End of Course assessments are developed, what alternative

assessments are needed? Should an alternative have different “entry” levels for the same

grade?


Subcommittee QuestionsFormative and Interim

What does it measure? What does it not measure? Should it measure growth? What is the purpose of interim and formative assessments? What isn’t

the purpose? If done well, what is the impact of a formative and interim assessment

system on student performance? Who is primary user of the results of interim and formative assessments:

students, parents, teachers, principal, district, state? What is the role of the state, district, principal, teacher in the

development, administration, and use of interim and formative assessments?

What is the right mix of commercial vs. local vs. state produced options? What is the process for vetting those options?

What are the roles and responsibilities of state and local education agencies in professional development for proper use of interim and formative assessments?


Subcommittee Meetings Publicly posted agendas and minutes from

all meetings

Held in public but not meetings of the public

Subcommittee member seats are non-transferable

You were selected as individuals not as representatives of an organization or place of employment


Roles and Responsibilities Assessment Subcommittee members

Commit to the scope of work Analyze research reports and exemplars Utilize public input Make informed recommendations about the design

of Colorado’s next generation assessment systemMembers should:

Be prepared to participate (attendance and completed homework) Open to new ideas Listen to seek understanding Look for ways to expand understanding Disagree productively Avoid defensiveness Be committed to deadlines and provide on-time delivery of product Build an assessment system worthy of our standards


Roles and Responsibilities Assessment Subcommittee Co-chairs

Responsible for the integrity of the revision process

Ensure timely delivery of the product: Facilitation (consensus-building, discussion leader)

Co-Chair tasks include:o Weekly meetings with facilitators (Phone or in-person)o De-brief from previous meetingo Prepare for upcoming meetingo Work with CDE to develop agendas, synthesize previous

meeting information, and discuss subcommittee questions and concerns


Subcommittee Pledge

Subcommittee Agreement: Members will research and study exemplars, consider options and reach consensus in order to make informed recommendations to the Assessment Stakeholders Committee about the design of Colorado’s next generation assessment system.


Subcommittee PledgeThe Assessment System Specifications are to be designed:• In a manner which aligns a cohesive and seamless P-20 system.• So that each component of the assessment system will be valid and reliable. • So that the new assessment system measures growth.• To measure mastery of the Colorado Academic Standards for all students which

includes the application of content through the demonstration of 21st century skills and readiness competencies, at appropriate grade levels.

• To signal postsecondary and workforce readiness of all students.• To reflect the belief that the opportunity to demonstrate success is a right for

all Colorado students. • So that the results are meaningful and accessible to all stakeholders –

students, parents, educators, higher education – and must inform instruction and interventions for teachers.

• To be signals for mastery and readiness, easy to use, meaningful, and timely. • To clearly articulate the purpose(s) of each assessment component and how

results will be used.


Subcommittee PledgeThe Assessment System Revision Process is intended to:• Ensure a process with integrity and honesty.• Make certain that the process is transparent and not commercial.• Respect the outcomes of the subcommittee and prevent sabotage.• Bring forth the voices of subcommittee members that represent their

expertise and merit rather than the ideas of the institution for which they work.

• Allow co-chairs to facilitate the members’ ideas in a way that leads to consensus and a rationale for each recommendation.

• Deliver a complete recommendation by the due date.

• Co-Chairs Only: Facilitate the members’ ideas in a way that leads to consensus and a rationale for each recommendation.


Subcommittee Meeting PlanMeeting 1 – July 17 Pueblo, CSU Pueblo“Landscape”

• Landscape: roles and responsibilities of all in the process, charge, norms, conceptual, background and context of an aligned system, understand standards.

• What are we measuring? What are we NOT measuring?• Housekeeping: pledge, reimbursements, future meeting logistics, etc.

Meeting 2 – August 14 – Denver DTC Doubletree“Get Specific”

• Discuss national activity – RTTT, Consortia, potential partners.• Hash out specific concepts of desired assessment – fill in template. • Consider research, exemplars, options paper, etc.• “Testing your assumptions” activity.• Homework: start thinking about consequences and benefits and how to utilize

data and results.


Subcommittee Meeting Plan Meeting 3 – August 28, Grand Junction Two Rivers Convention Center“Consider Feedback and Alignment”

• Discuss matrix prepared by facilitators for alignment purposes. • Conduct joint subcommittee sessions to find alignment through consensus.• Identify roles and responsibilities of students, teachers, districts, CDE, higher

education.• Consequences and benefits: what are the stakes and incentives?• Homework: alignment issues resolved and non-negotiables.

Meeting 4 – September 11, Fort Collins Hilton“Alignment and Deliver the Final Recommendations”

• Resolve any remaining alignment issues:– Matrix prepared by all facilitators for alignment purposes – Joint subcommittee sessions to find alignment through consensus

• Consideration of TAC recommendations.• Timely delivery of product.


Roles and ResponsibilitiesColorado Department of Education

Facilitators Serves as assessment resources to the

subcommittees– Experience in subcommittee specific work– Template resource guide– Assimilates subcommittee meeting

information Continuity and Alignment advisors

– Between and within subcommittee work– Matrix design

Process Feedback


Roles and ResponsibilitiesColorado Department of Education

Office of Standards, Assessments and Research & Evaluation CAP4K Project Manager Creates the design of the review process Communicates updates to ensure transparency Delivers recommendations to SBE Ensures alignment Technical support Planning and logistics


CDE Support and Direction for Subcommittee Work

CDE directed

Co-chair directed

Meeting 1 Meeting 4


Roles and Responsibilities Technical Advisory Committee (TAC)

Provides technical response only Advisory role to CDE in the design of the

assessment system revision Review draft recommendations from the third

subcommittee meetings– Consistency/Alignment, Validity, Peer Review-able

Review final recommendations from the Stakeholders – Consistency/Alignment, Validity, Peer Review-able


Roles and Responsibilities WestEd

Nationally recognized third-party Standards and Assessment research and review Produce Assessment Options Paper Consult on revision process Conduct Assessment Capacity Study Support Stakeholders and subcommittees


Roles and Responsibilities Colorado Department of Higher Education

Equal Partner in the development of the Postsecondary and workforce readiness assessment

On-going resource and liaison to institution of Higher Education

Convene and gather feedback from: Provosts C-CODE Admissions Chief Academic Officers Student Leaders

Co-Facilitators of two 13-city feedback tours


Roles and Responsibilities Colorado State Board of Education

Adopt new Colorado Assessment System Co-adopt the Colorado Postsecondary and

Workforce Readiness Assessment with the Colorado Commissioner on Higher Education


Roles and Responsibilities Colorado Commissioner on Higher

Education Co-adopt the Colorado Postsecondary and

Workforce Readiness Assessment with the Colorado State Board of Education


Timeline June 3 - State Board of Education Update on Assessment Development10 - Assessment Subcommittees Conference Call/Webinar

July9 - Assessment Stakeholders Advisory Committee meeting17 - Assessment Subcommittees Meeting no. 1

August6 - Assessment Stakeholders Advisory Committee meeting11 - State Board of Education Update on Assessment Development14 - Assessment Subcommittees Meeting no. 228 - Assessment Subcommittees meeting no. 3


Timeline September 8 - Technical Advisory Committee (TAC) meeting11 - Assessment Subcommittees meeting no. 415 - State Board of Education Update on Assessment Development20 - Assessment Stakeholders Advisory Committee meeting22 - Technical Advisory Committee (TAC) meeting (Conference Call)24 - Assessment Stakeholders Advisory Committee conference call27-30 - Joint CDE and DHE state-wide tour begins

October1-7 - Joint CDE and DHE state-wide tour continues6 - State Board of Education Update on Assessment Development15 - Assessment Stakeholders Advisory Committee Meeting


Timeline

November 10 - State Board of Education meeting – present final recommendations for Colorado’s new assessment system design specifications and invite public comment.

December8 - State Board of Education considers adoption new assessment system specs.


Context of Aligned System WestEd Options Paper Introduction Big Ideas


Assessment System Revision Process

What questions do you have? What do we need to discuss further?


• Save the Dates:

August 14 – Denver Doubletree Denver Tech Center, 7801 East Orchard Rd., Greenwood Village, CO 80111-2508Please make reservations before July 23 to receive the government rate! Phone: 303-779-6161

August 28 – Grand JunctionMeeting: Two Rivers Convention Center, 159 Main St., Grand Junction, CO 81501Hotel: Doubletree Hotel Grand Junction, 743 Horizon Dr, Grand Junction, CO 81506-3906. Please call: 970-241-8888 to make your hotel room

reservation before August 4 to receive the government rate!

September 11 – Fort CollinsThe Hilton Fort Collins, 425 West Prospect Road, Fort Collins, Colorado 80526Phone: 970-482-2626

welcome!

Documents