the quality matters™ k-12 program overview. the quality matters program quality assurance through...

The Quality Matters™ K-12Program Overview

The Quality Matters Program

Quality Assurance through Faculty Development and Course Design

© 2014 MarylandOnline, Inc.. All rights reserved

Academic & Non-Profit

• Quality Matters is a not-for-profit subscription service providing tools and training for quality assurance of online courses

• Developed by MarylandOnline with funding from FIPSE, it was designed by faculty for faculty and focused on improving student learning

• Adopted by a large and broad user base, QM represents a shared understanding of quality in online course design


Quality Matters is…

• A set of standards (Rubric) for the design of online and blended courses

• A peer review process for reviewing and improving online and blended courses

• A faculty support tool used by instructional development staff

• A professional development opportunity© 2014 MarylandOnline, Inc.. All rights reserved

The QM Approach

• QM is a faculty-driven, peer review process that is…

CollaborativeCollegialContinuousCentered

• in academic foundation• around student learning


Factors Affecting Course Quality

QM Only reviews Course Design © 2014 MarylandOnline, Inc.. All rights reserved

QM’s Role in Quality Assurance

• QM looks at course design– The harnessing of technology to deliver instruction and promote

student learning

• QM provides a process for peer-to-peer feedback for faculty in the continuous improvement of their course and certifies courses as meeting shared standards of best practice

• Quality Matters is not the complete answer to quality assurance for online education, but it can be a critical component


What QM is NOT About…

• Not about an individual instructor(it’s about the course)

• Not about instructor evaluation(it’s about course quality)

• Not about judgment(it’s about diagnosis and improvement)

• Not about “win/lose” or “pass/fail”(it’s about continuous improvement in a supportive environment)


QM as a National Standard

• 800+ current subscribers• 46 states represented• QM has trained over 22,000+ Higher Ed and K-12

instructors and instructional design staff• Community Collaboration

– Largest community collaboration in quality online education

• Award Winning– Sloan-C, USDLA, WCET


MONTANA

WYOMING

IDAHO

WASHINGTON

OREGON

NEVADA

UTAH

CALIFORNIA

ARIZONA

NORTH DAKOTA

SOUTH DAKOTA

NEBRASKA

COLORADO

NEW MEXICO

TEXAS

OKLAHOMA

KANSAS

ARKANSAS

LOUISIANA

MISSOURI

IOWA

MINNESOTA

WISCONSIN

ILLINOISINDIANA

KENTUCKY

TENNESSEE

MISS

ALABAMA GEORGIA

FLORIDA

SOUTHCAROLINA

NORTH CAROLINA

VIRGINIA

WV

OHIO

MICHIGANNEW YORK

PENN

MARYLAND

DELAWARE

NEWJERSEY

CONNRI

MASS

MAINE

VTNH

ALASKA

HAWAII

PUERTO RICO

VIRGIN ISLANDS

Current Subscribers

Consortium Subscribers

QM as a National Standard

Statewide Subscribers


QM BasicsMore than a sum of its three parts

1 2 3


• QM collaborated with The Florida Virtual School (FLVS) to create the Grades 6-12 Rubric.

• The Grades 6-12 Rubric integrates existing national standards for K-12 online education promulgated by the following organizations:• International Society for Technology in Education (ISTE)• The Southern Regional Education Board (SREB)• The International Council for K-12 Online Learning (iNACOL)• The Partnership for 21st Century Skills

• The Grades 6-12 Rubric was revised in 2013 and renamed “The K-12 Secondary Rubric Second Edition.”

QM K-12


The QM K-12 Secondary Rubric


Audiences for K-12 Secondary Rubric

Users Applications

Teachers, Instructional Designers and Design Teams

Improve existing courses and design more effective courses

Public and Private Virtual Schools and Publishers

Demonstrate commitment to quality to school systems and other stakeholders

Schools Administrators Standards to screen and select courses

Students, Parents and Counselors

Assurance of quality courses for students

School Systems, Local and State Boards

Component of quality assurance mission

Teachers of Dual Enrollment Courses

Meeting both higher education and Grades 6-12 standards

Teacher Education Faculty Preparing their students to teach online


The Rubric is the Core of QM

8 key areas (general standards) of course quality and an additional general standard to customize for local course mandates

42 specific review standards• Including 27 essential (3 point) standards• Detailed annotation and examples of good practice for all 42

standards• Plus a customizable Standard 9

104 point scoring system • 88 points needed to meet standards


Elements of the Rubric

Eight General Standards:

1. Course Overview and Introduction2. Learning Objectives (Competencies)3. Assessment and Measurement4. Instructional Materials5. Learner Interaction and Engagement6. Course Technology7. Learner Support8. Accessibility9. Compliance Standards

Key components must align.

Alignment: Critical course elements work together to ensure that students achieve the desired learning outcomes.


General Standard 9: Compliance

• This section of the Rubric is optional and may be tailored to particular requirements or mandates at the state or local level.

• These requirements may deal with subject matter standards, inclusion of specific information in the course outline or syllabus, mandated communications, etc.

• Up to 10 standards may be added in this section.

• No point value is associated with these standards, but the review summary will identify any that were not considered as met by the review team.


QM Rubrics: K-12 and K-12 Publisher

K-12 Secondary K-12 Publisher

• Local teachers• Course developers

• Review courses• Develop courses

• Vendor Courses• Locally developed

courses for use in other districts

• Vetted courses


Standards Supported by Research


QM Course Review Process

1 2 3


The QM Course Review Process

Course Continues to be Offered


Types of Course Reviews

Unofficial Team Reviews

Self Review Using QM

Tools

OfficialQM Managed

Review



Unofficial Team Reviews*

Self Review Using QM

Tools Unofficial reviews are not required to follow the formal QM process or use the entire rubric



OfficialQM-Managed

Review

• Follows QM policies/protocols • Fee-based• QM manages and pays review team

It is recommended that staff go through appropriate QM training prior to starting a QM review.


QM = Process and Rubric

Process Rubric

OFFICIAL Outcome: Earn QM recognitionMust follow official QM guidelines and procedures

Outcome: Improve courses, meet institutional goals, demonstrate commitment to quality

Outcome: Earn QM recognitionTool to assess online courses during formal QM review

Outcome: Improve courses, meet institutional goals

Guide to develop new online courses AND review and update online courses

UNOFFICIAL Outcome: Improve courses, meet institutional goals, demonstrate commitment to quality

Organizations determine use and procedures

Outcome: Improve courses, meet institutional goals, demonstrate commitment to quality

Guide to develop new online courses AND review and update online courses


The Course Review Team

On an official K-12 review, Review Teams consist of:

• Three QM-Certified K-12 Course Reviewers• Pre-requisite: Current online teaching or development experience & training

• Team Chair• Course Reviewer with additional experience

• Two reviewers must be a subject matter experts• At least one reviewer must be external to the organization

sponsoring the course

AND • Course Developer or Representative:

• access to rubric prior to review• involved in pre-review discussions• consulted during review


The Course Review Team

On an official K-12 PUBLISHER review, Review Teams consist of:

• Three QM-Certified K-12 Course Reviewers• Pre-requisite: Current online teaching or development experience &

training• Master Reviewer as Team Chair

• Course Reviewer with additional experience and training• Two reviewers must be a subject matter expert

AND • Course Developer or Representative:

• access to rubric prior to review• involved in pre-review discussions• consulted during review


Peer Feedback

The peer feedback provided in a course review is collegial in tone and language; however, the reviews are rigorous and even courses that meet standards will benefit from the detailed, specific, and relevant feedback provided.


QM Certification of Courses

• Course reviews take 4-6 weeks; 20-week window for reviews, including any amendments, to be completed

• QM recognition is provided by year recognized • QM logo on course and catalog; registry

on QM Web site


Professional Development and Certification

QM Workshops & Certifications


QM Workshops & Certifications

Delivery Options:• Online – virtual, asynchronous• Onsite – face-to-face, synchronous• Collaborate – virtual, synchronous

Focus:• QM Implementation

• Prepare staff to hold QM roles • Professional Development Workshops

• Prepare staff to design and improve courses


Professional Development Workshops

Course-focused Workshops• K-12 Applying the QM Secondary Rubric*• K-12 Online Course Design• Flipping Your Class and More: Exploring Blended

Learning• K-12 Introduction to Teaching Online

*Note: Flagship training is also required for QM Implementation


QM Implementation: Certifications

• K-12 Reviewer Course (K-12 RC)*• K-12 Publisher Reviewer Course (K-12 PubRC)*• K-12 Master Reviewer Certification (K-12 MRC)*• K-12 Online Facilitator Certification (K-12 OFC)*

• For online delivery of K-12 APP• K-12 QM Coordinator Training*

*Offered online only


QM Coordinator

K-12 QM Coordinator Training

K-12 Applying the QM Secondary

Rubric Workshop (online or F2F)

Organization may designate additional

QMCs

K-12 Course Reviewer

Taught or developed an online course within the last 18

mo.



K-12 Reviewer Course (online)

Submission of application & MOU

(for certification)

K-12 Publisher Reviewer

Taught or developed an online course within the last 18

mo.



K-12 Reviewer Course (online)

K-12 Publisher Reviewer Course

(online)

Submission of application & MOU

(for certification)

K-12 Certified Facilitator


Rubric Workshop

K-12 Reviewer Course (certification

not required)

K-12 Online Facilitator

Certification Course

Only available with purchase of a K-12

Training License

K-12 Master Reviewer

Certified K-12 Course Reviewer

Experience on 2+ course reviews

K-12 Master Reviewer

Certification Course

Annual Recertification

Capacity Building: QM Roles & Criteria


The Quality Matters Subscription

What can you gain?


What can you gain?

• Improvement of online courses

• External quality assurance

• Participation useful for professional development plan and portfolio

• Professional development trainings

• Review other courses and gain new ideas for own course; expand professional community


Tools in the Spirit of the Academy

• Standards grounded in the research literature and best practices

• Tools and process developed by faculty and instructional design experts for faculty support

• Peer review and other opportunities to collaborate and mentor around quality in online learning


What to Expect from Course Reviews

• Consistency and Rigor

• Professionalism and Commitment

• Useful and Constructive Feedback

• Positive Outcomes

• Less than 50% of courses in QM-managed reviews meet standards upon initial review (but all can meet after amendment)


Review Exit Survey Comments

Sample Comments from Course Developers

• “I was too close to see what could be improved.”

• “Provides a great way to get an objective view of your course.”

• “It made all of my online courses better.”

• “It provides a view from a more student oriented perspective.”

• “Many elements that might contribute to a student withdrawing can be eliminated.”


Review Exit Survey Comments

Sample Comments from Course Reviewers

• “I always learn new wonderful approaches from viewing other courses and working with expert peer reviewers.”

• “Whenever I read the annotations, I make changes to my courses.”

• “I always find reviews to be a learning experience.”

• “This experience was very rewarding for me as both an educator and course developer.”

• “It’s a great tool for honing your own course.”


Research Opportunities and Support

• QM encourages research through

– Consultation with QM Director of Research

– Collaboration on research and data resources

– Participation in faculty surveys of all QM subscribers

– Sponsorship of research - see RFP at www.qmprogram.org


Quality Matters: A Multi-faceted Resource

What’s In It for your organization?


Multiple Uses of QM

Reported Uses of QM Tools:• Guidelines for initial online course development• Quality assurance of existing courses• Ongoing professional development• Institutional re-accreditation packages• Focus attention on distance learning policies &

steering committees• Create a climate and structure that promotes

teaching and learning


Reported uses:• Educate faculty about quality in online courses • Reduce time and labor in designing courses.• Provide professional development opportunities

for faculty and staff• Inform teaching practices related to design and

structure for BOTH online and F2F courses• Quality process indicators for accreditation self-

studies

QM Professional Development


Subscription Benefits

• Current QM Rubric and related rubrics• Fully automated QM Rubric Tool(s)• Subscriber Discounts• QM Coordinator Training• MyQM & User’s Group• Unofficial Course Reviews• Training License


Quality Matters: A Collaboration on Quality

The QM Community


The QM Community

Online Learning Students

QM Institutions: HE, K-12, Publishers, Partners, Sponsors

Committees: Academic Advisory, Rubric, Training, Reviews

QMCs, Facilitators, Peer/Course Reviewers

QM Staff& Consultants


QM is Continuous Improvement

• Improved courses• Engaged faculty• Reduced course

development time• Ongoing professional

development• Quality benchmarking• Awareness & support for

online learning• Institutional

improvement© 2014 MarylandOnline, Inc.. All rights reserved

Your Questions?

Please contact the appropriate QM representative – available to you via

email, Skype or phone.


We are here for you! QM Contacts:

• Christine Voelker, K-12 Program [email protected]

• Maggie Bacon, K-12 Program [email protected]


mailto:[email protected]



the quality matters™ k-12 program overview. the quality matters program quality assurance through...

Documents

course quality qm

rights reserved slide

quality assurance qm

qm approach qm

online course design

qm basics

student learning qm

design of online