the ontario mathematics proficiency test (mpt)

THE ONTARIO

MATHEMATICS PROFICIENCY TEST (MPT)

ASSESSMENT BLUEPRINT

© Education Quality and Accountability Office 2020

of 27

Contents Overview ........................................................................................................................ 4

Demonstrating Mathematics Proficiency ...................................................................... 4

Underpinning Principles ............................................................................................... 4

Components of the Mathematics Proficiency Test ....................................................... 5

Big Ideas in Mathematics ............................................................................................. 5

Assessment Model ....................................................................................................... 6

Number of Items ....................................................................................................... 6

Time Period .............................................................................................................. 6

Framework for Mathematics Content .......................................................................... 7

Content Dimensions for Mathematics .......................................................................... 7

Subcategories .............................................................................................................. 7

Number Sense .......................................................................................................... 7

Relationships and Proportional Reasoning ............................................................... 7

Measurement ............................................................................................................ 7

Categories of Knowledge and Skills ............................................................................. 8

Financial Literacy ......................................................................................................... 8

Distribution of Assessment Items ................................................................................. 9

Distribution of Mathematics Content Dimensions ..................................................... 9

Distribution of the Categories of Knowledge and Skills ........................................... 10

Distribution of Mathematics Content Dimensions within the Categories of

Knowledge and Skills .............................................................................................. 10

Distribution of Mathematics Content Assessment Items ......................................... 11

Framework for Pedagogy ........................................................................................... 12

Content Dimensions for Pedagogy ............................................................................ 12

Growing Success .................................................................................................... 12

Learning for All........................................................................................................ 12

Front Matter of the Mathematics Curriculum ........................................................... 12

Distribution of Assessment Items ............................................................................... 13

Distribution of Pedagogy Dimensions ..................................................................... 13

Distribution of Pedagogy Assessment Items .......................................................... 13

of 27

Assessment Items and Questionnaire ...................................................................... 14

Item Types ................................................................................................................. 14

Bias and Sensitivity .................................................................................................... 15

Non-Calculator Items and Calculator-Enabled Items ................................................. 15

Item Bank ................................................................................................................... 15

Item Pre-testing .......................................................................................................... 16

Item Field Testing ...................................................................................................... 16

Questionnaire Items ................................................................................................... 16

Test Design .................................................................................................................. 17

Test Design Criteria ................................................................................................... 17

Assessment Delivery Method ..................................................................................... 17

Construction of Test Forms ........................................................................................ 18

Mathematics Content Component .......................................................................... 18

Pedagogy Component ............................................................................................ 21

On-Demand Test Form Creation ................................................................................ 22

Reporting ................................................................................................................... 22

Accommodation and Accessibility ............................................................................ 24

Accommodation ...................................................................................................... 24

Accessibility ............................................................................................................ 24

Data Analysis and Security ........................................................................................ 25

Data Analysis ............................................................................................................. 25

Security ...................................................................................................................... 25

References ................................................................................................................... 26

of 27

Overview

Demonstrating Mathematics Proficiency

The Ontario government requires all teacher applicants to demonstrate their

mathematics proficiency through the successful completion of a mathematics

proficiency test. Accordingly, the Ontario Legislature has enacted, as part of the Safe

and Supportive Classrooms Act, 2019, amendments to the Ontario College of Teachers

Act (OCTA), 1996, that all Ontario applicants (individuals who are enrolled in, or have

completed, a program of professional education in Ontario) and Internationally

Educated Teacher applicants must successfully complete the Mathematics

Proficiency Test (MPT) prior to qualification. See O. Reg. 271/19: Proficiency in

Mathematics for information on teacher applicant exemptions.1

The principal purpose of the MPT, according to the Ontario Deputy Minister of

Education, is to “enhance teacher confidence and sense of efficacy in teaching

mathematics, which will ensure that students are better prepared for success in all

aspects of their lives”.

Subsequently, the Ontario Ministry of Education (OMOE) has mandated the Education

Quality and Accountability Office (EQAO) to take responsibility for the development and

reporting of the MPT and has issued regulations concerning aspects of the form,

content, and operations of the test, details of which are set out in this Blueprint.

Underpinning Principles

The philosophy of assessment at the Ontario Ministry of Education and at EQAO has always been that the primary purpose of assessment is to support learning. This philosophy underpins the development and operation of the MPT.

The MPT is not intended to discourage new teachers from entering the profession nor is

it intended to add difficulties to teacher education either for teacher applicants or

Ontario Faculties of Education (OFOE). It is rather, as the Ministry of Education has

mentioned, a means to “enhance teacher confidence” and to support teachers in a

subject area where their confidence may be low. Accordingly, the MPT is largely based

on the principles of Assessment for Learning.

The MPT combines the diagnostic assessment of teacher applicants’ strengths and

weaknesses in mathematics content and pedagogy with feedback and opportunities for

remediation in those areas shown to be needed. Teacher applicants can reattempt the

test to build confidence and confirm their proficiency in mathematics.

1 O. Reg 271/19: Proficiency in Mathematics under Ontario College of Teachers Act, 1996, S.O. 1996, c. 12. Retrieved from https://www.ontario.ca/laws/regulation/190271

https://www.ontario.ca/laws/statute/96o12


of 27

Components of the Mathematics Proficiency Test The regulations of structuring the MPT call for the test to be composed of two

components:

• A mathematics content component (70% of the test) based on mathematics

content from the Ontario Curriculum Mathematics documents (Grades 3 to 9),

and

• A pedagogy component (30% of the test) based on foundational understanding

of the mathematics curriculum, assessment and evaluation practices, as well as

learning for all students, centred on Ministry policy documents:

✓ Growing Success: Assessment, Evaluation, and Reporting in Ontario

Schools - First Edition, Covering Grades 1 to 12, 2010 (Growing Success),

✓ Learning for All: A Guide to Effective Assessment and Instruction for All

Students, Kindergarten to Grade 12, 2013 (Learning for All), and the

✓ Front matter of the Ontario Curriculum Mathematics documents.2 This

refers to the pages of each curriculum document that precede the

curriculum expectations for the specified grades (Front Matter).

Big Ideas in Mathematics The MPT is not intended to assess all the mathematics skills and knowledge from

Grades 3 through 9. The assessment items that comprise the mathematics content

component of the MPT are based on core understandings, otherwise known as big

ideas, surrounding important content dimensions in mathematics. Big ideas in

mathematics involve the interconnectedness of concepts “that form a framework for

learning mathematics in a coherent way” (Ontario Ministry of Education, 2005a, p.4).

The MPT is developed on the principle that it is critical for teachers to have a good

grasp of how mathematical concepts are interconnected through overarching big ideas.

“Teachers need to understand the big ideas of mathematics and be able to represent

mathematics as a coherent and connected enterprise” (Martin & Herrera, 2010, p.16).

The blueprint for the MPT involves overarching big ideas based on the following

important mathematics content dimensions, as reflected in the Ontario curriculum:

✓ Number Sense,

✓ Relationships and Proportional Reasoning, and

✓ Measurement.

Assessment items are embedded in contexts that reflect big ideas of the above three

mathematics content dimensions so that teacher applicants can demonstrate

proficiency in key principles of mathematics knowledge and apply mathematical

reasoning (Ontario Ministry of Education, 2005b).

2 Ontario Ministry of Education. (2005a). The Ontario Curriculum, Grades 1-8: Mathematics; Ontario Ministry of Education. (2005b). The Ontario Curriculum, Grades 9-10: Mathematics.

of 27

Assessment Model

Teacher applicants attempt the MPT in a proctored environment, using a computer-

based testing (CBT) delivery method. They can select a test session (from a test

window) on the platform according to their choice, based on timing, location, available

test sessions, and required accommodation. (Note: Test sessions are time periods that

are pre-set by test administrators for teacher applicants to attempt the MPT. Test

windows are time periods pre-set by EQAO for test administrators to hold test sessions.

EQAO has planned to have 4 one-month test windows every year: winter, spring,

summer, and fall.)

The MPT is automatically machine-scored and generates diagnostic feedback on the

performance to the teacher applicants within 10 business days of completion of the

test.3 The feedback provides teacher applicants with their overall performance on both

the mathematics content and pedagogy components of the test, and whether they were

successful in each component. Additionally, teacher applicants receive feedback on the

mathematics content dimensions and the associated categories of knowledge and skills,

to support in identifying areas of the assessment where they hold a strong

understanding and areas of the assessment where they need to remediate.

Upon completion of the assessment and receiving feedback, teacher applicants have

the opportunity to upgrade their skills. They can reattempt the MPT as many times as

they like, or as needed, based on test session availability.

Number of Items

The MPT has 75 items. Each unique test (test form) contains 71 items that count

towards the teacher applicant’s score. These are 50 mathematics content items (70% of

the test) and 21 pedagogy items (30% of the test).

The first 5 mathematics content items will cover the Knowledge and Understanding of

Number Sense. Teacher applicants are required to solve these items without the use of

a calculator. The remaining 45 mathematics content items can be answered with the

use (if desired) of a built-in calculator on the platform. The mathematics content and

pedagogy items are drawn from a test bank of psychometrically validated assessment

items.

In every test administration, 4 field test items (that do not count towards the teacher

applicant’s score) are included. These are 1 non-calculator item, 2 calculator-enabled

items, and 1 pedagogy item.

Time Period

The MPT is scheduled for 3-hour test windows.4 Some items may take a few seconds to answer, while more complex items may take longer.

3 Results are immediately made available for EQAO to review and validate prior to the report being automatically emailed from the system to the respective candidate.

4 Additional accommodation can be provided if required. See Accommodation and Accessibility section.

of 27

Framework for Mathematics Content

Content Dimensions for Mathematics The mathematics content component of the MPT is designed to assess the

understanding of big ideas based on the three mathematics content dimensions that

encompass a wide range of learning expectations from Grade 3 to Grade 9 within the

Ontario Mathematics Curriculum. The mathematics content dimensions are Number

Sense, Relationships and Proportional Reasoning, and Measurement.

The content of the assessment items is embedded in contexts that reflect big ideas in

mathematics. This enables teacher applicants to demonstrate their understanding of

key fundamental principles in mathematics and the application of mathematical

reasoning (OMOE, 2005b, p.23).

Subcategories The subcategories of the mathematics content dimensions are interrelated and

represent fundamental mathematics concepts. The assessment items in the MPT cover

all the subcategories to ensure that a balance of concepts is assessed. The blueprint for

the mathematics content component of the MPT includes the following subcategories:

Number Sense • Operating with whole numbers, integers, decimals, and fractions: addition, subtraction,

multiplication, division, and square root

• Expressing whole numbers using place value and expanded form

• Working with exponents

• Evaluating numeric expressions involving order of operations

Relationships and Proportional Reasoning • Ordering and comparing whole numbers, integers, decimals and fractions

• Using relationships among fractions, decimals, and percent

• Working with percent

• Working with ratios, proportions, rates, and unit rates

• Solving problems involving proportional thinking

• Solving problems involving probability

• Recognizing linear vs non-linear relationships

• Representing linear relations: graphically, numerically, and algebraically

• Solving first-degree equations

• Solving linear systems graphically

• Solving mean, median, and mode

Measurement (formulas provided) • Applying Pythagorean theorem

• Solving problems involving conversions of metric units (perimeter, area, and volume).

• Solving problems involving perimeter of regular and irregular shapes and circles

• Solving problems involving area of regular and irregular shapes and circles

• Solving problems involving surface area of prisms and cylinders

• Solving problems involving volume of prisms and cylinders

of 27

Categories of Knowledge and Skills

The mathematics content assessment items are categorized under one of the three categories of knowledge and skills:

• Knowledge and Understanding: Assesses the basic knowledge and understanding of mathematics concepts.

For example, demonstrating an understanding of the tenths place value in decimal numbers.

• Application: Assesses the use of knowledge and understanding to make connections within and between various contexts (OMOE, 2005a).

For example, applying the relationship between area and perimeter to solve for the area of a shape.

• Thinking: Assesses critical thinking processes.

For example, the use of critical thinking to find the surface area of two attached cubes.

A higher proportion of assessment items are mapped to the Application category of

knowledge and skills because the MPT is a minimum-competency mathematics

assessment and teacher applicants need to demonstrate that they can apply their

knowledge and understanding of mathematics concepts to various contexts, using a

variety of tools and strategies to solve problems.

Financial Literacy

Through mathematics concepts, financial literacy engages major themes such as

personal finances, budgeting, consumer literacy, and economic contexts (OMOE,

2016a; OMOE, 2016b). At least 12 mathematics content items (24%+) of the

mathematics content component of the MPT have a financial literacy context. These

items cover content from any of the mathematics content dimensions and any of the

categories of knowledge and skills.

Money amounts and financial literacy are embedded in specific mathematical learning

expectations seen throughout The Ontario Mathematics Curricula. We can see this

starting in Grade 3 Number Sense and Numeration, where students are expected to

“represent (using the $ symbol) the value of a collection of coins” (OMOE, 2005a, p.55),

and all the way to Grade 9.

Financial literacy is not only important when it comes to the Ontario Curriculum

expectations but is embedded in the daily lives of people. The MPT covers financial

literacy in day-to-day contexts, such as earning and purchasing, saving, investing and

borrowing, and scenarios related to transportation and travel.

of 27

Distribution of Assessment Items

This section describes the distribution of the mathematics content dimensions and

categories of knowledge and skills that make up the mathematics content component

(70% of the MPT) of test forms.

Distribution of Mathematics Content Dimensions

The MPT is designed to assess the Knowledge and Understanding, Application, and Thinking strategies of big ideas in mathematics based on the following mathematics content dimensions: Number Sense, Relationships and Proportional Reasoning, and Measurement.

The Ontario Curriculum, Grades 1-8: Mathematics states, under the Number Sense and Numeration strand, that “a well-developed understanding of number includes a grasp of more-or-less relationships, part-whole relationships, measures in the environment, and much more” (OMOE, 2005, p.8). The document also states that representing number sense in various contexts can help students to construct an understanding of numbers.

Measurement units are often used within a proportional reasoning context (when

examining rates and ratios), also involving Knowledge and Understanding of Number

Sense.

All the mathematics content dimensions are connected. However, a larger overlap of

Number Sense and Measurement is shown within relationships and proportional

reasoning.

The following table shows the distribution of the mathematics content dimensions on the

MPT.

MATHEMATICS CONTENT DIMENSIONS DISTRIBUTION

Number Sense 32%

Relationships and Proportional Reasoning 54%

Measurement 14%

of 27

Distribution of the Categories of Knowledge and Skills

As mentioned previously, the highest proportion of assessment items is mapped to the Application category of knowledge and skills. Since the MPT is a minimum-competency mathematics assessment, teacher applicants need to demonstrate that they can apply their understanding of mathematics concepts to various contexts.

As with the distribution of mathematics content dimensions, there is an overlap between the categories of knowledge and skills. A variety of items involve Knowledge and Understanding, Application, and Thinking.

Knowledge and Understanding is demonstrated within Application items and within

Thinking items. Teacher applicants need Knowledge and Understanding in order to

apply mathematics to a context as well as to plan, process, and critically think about

how they will solve a problem. Items mapped to the skill of Thinking are more complex

than those mapped to Knowledge and Understanding or Application. The cognitive load

on the teacher applicant is greater for Thinking items. The MPT has a larger proportion

of items mapped to Knowledge and Understanding and Application than to Thinking.

The following table shows the distribution of categories of knowledge and skills on the

MPT.

CATEGORIES OF KNOWLEDGE AND SKILLS DISTRIBUTION

Knowledge and Understanding 40%

Application 50%

Thinking 10%

Distribution of Mathematics Content Dimensions within the Categories of

Knowledge and Skills

The mathematics content component of MPT consists of a total of 50 assessment

items.

The Ontario Curriculum, Grades 1 to 8: Mathematics states that the Number Sense and

Numeration strand refers to a general understanding of number and operations as well

as to the ability to apply this understanding in flexible ways (OMOE, 2005a). Therefore,

Number Sense is assessed through items mapped to Knowledge and Understanding

and Application.

Relationships and Proportional Reasoning is assessed across all categories of

knowledge and skills with an emphasis towards Knowledge and Understanding and

Application mapped items, but still embedded among a variety of Thinking items.

of 27

Measurement is directly applicable to real world, contextualized problem solving

associated with telling time, finding length, perimeter, area, etc. (OMOE, 2005a). Thus,

Measurement knowledge and skills are assessed through Application and Thinking

items.

The following table shows the distribution of mathematics content assessment

dimensions within the categories of knowledge and skills on the MPT.

MATHEMATICS CONTENT DIMENSIONS KNOWLEDGE AND UNDERSTANDING

APPLICATION THINKING

Number Sense 20% 12% 0%

Relationships and Proportional Reasoning

20% 28% 6%

Measurement 0% 10% 4%

Distribution of Mathematics Content Assessment Items

The table below shows how the types of assessment items are distributed among the

mathematics content dimensions and categories of knowledge and skills, ultimately

showing the number of assessment items in the mathematics content component of the

MPT: 16 number sense items, 27 relationships and proportional reasoning items, and 7

measurement items.

From the table, it is also evident that a greater proportion of number sense items require

Knowledge and Understanding, whereas a greater percentage of relationships and

proportional reasoning and measurement items require an application component,

showing a framework that has been further described in the following sections of the

Blueprint.

The following table shows the distribution of mathematics content items on the MPT.

MATHEMATICS CONTENT DIMENSIONS KNOWLEDGE AND UNDERSTANDING

APPLICATION THINKING

Number Sense 10 Items 6 Items 0 Items

Relationships and Proportional Reasoning

10 Items 14 Items 3 Items

Measurement 0 Items 5 Items 2 Items

of 27

Framework for Pedagogy

Content Dimensions for Pedagogy

The pedagogy component assesses knowledge surrounding Growing Success,

Learning for All, and the Front Matter.

Growing Success

The Ontario Ministry of Education has outlined policies for the growing success of

Ontario students in regard to the assessment, evaluation, and reporting of students

across Ontario and within Ontario School Boards (2010). One of the primary roles of a

teacher is to assess and evaluate student achievement levels and to then implement

action to support the continued and growing success of students. Thus, this dimension

of the pedagogy component explores underlying principles, learning skills, working

habits, performance standards, knowledge of assessment (for, as and of learning),

knowledge of evaluation, and knowledge regarding the reporting of student

achievement, as outlined within Growing Success.

Learning for All

This section of the pedagogy portion assesses the knowledge of teacher applicants

regarding learning for all students. This includes but is not limited to the connection

between assessment and student achievement closely connected to many policies and

initiatives dealing with numeracy strategy, student success and equity, and inclusive

education strategies (Learning for All, 2013, p.5).

This dimension of the pedagogy component assesses the knowledge of instructional

approaches including differentiated instruction practices that supports learning for all

students across Ontario. This includes the strategies that teachers may use to help all

students reach their highest achievement potential ultimately leading towards closing

the achievement gap (2013). It is imperative that teacher applicants have knowledge of

effective instructional practices and strategies that support the learning of all students

across the province. This dimension of the pedagogy component focuses largely on the

assessment for student learning.

Front Matter of the Mathematics Curriculum

This dimension of the pedagogy component of the MPT focuses on the Front Matter,

ensuring that teacher applicants have an understanding of the underlying concepts,

ideas, program planning, strands, and the language used in the Ontario Mathematics

Curriculum. This dimension ensures that teacher applicants have a good understanding

of the pages of the curriculum that precede the curriculum expectations from Grade 3 to

Grade 9.

Assessing teacher applicants on pedagogy provides assurance that teachers are

effective in supporting Ontario student achievement levels within the mathematical

learning space across Ontario School Boards.

of 27

Distribution of Assessment Items

This section describes the distribution of the pedagogy dimensions that make up the

pedagogy component (30% of the MPT) of test forms.

Distribution of Pedagogy Dimensions

The three dimensions that make up the pedagogy component of the MPT are

distributed to place a greater emphasis on the first two dimensions: Growing Success

and Learning for All.

It is also understood that ideas and contents of each of these categories are intertwined

and connected. For example, assessment and evaluation of student achievement are

contents of both Growing Success and the Front Matter, whereas instructional

approaches are addressed in both the Learning for All and Front Matter. Therefore,

assessment items are mapped to the most appropriate category.

The following table shows the distribution of pedagogy dimensions in the MPT.

PEDAGOGY DIMENSIONS DISTRIBUTION

Growing Success 43%

Learning for All 43%

Front Matter of the Ontario Mathematics Curriculum 14%

Distribution of Pedagogy Assessment Items

The pedagogy component of the MPT consists of a total of 21 assessment items.

The following table shows the distribution of pedagogy items on the MPT.

PEDAGOGY DIMENSIONS NUMBER OF

ASSESSMENT ITEMS

Growing Success 9

Learning for All 9

Front Matter of the Ontario Mathematics Curriculum 3

of 27

Assessment Items and Questionnaire

Item Types

Assessment items are created in the form of multiple-choice questions and may include

the following types (e.g., checkboxes, drag and drop, selection, ordering lists, number

lines, etc.).

A few examples of the assessment item types include:

These types of items allow for immediate machine scoring of the results and can

therefore provide a list of successful teacher applicants to the OCT in a relatively quick

time period.

Number Line Drag and Drop Response Type

Drag and Drop Response Type

Selection Response Type Ordering Response Type

This number line segment is divided into four equal parts.

0 1

What is the missing value in the proportion? Drag the appropriate value into the indicated location on the number line.

Place the following measurements in order from least to greatest.

of 27

Bias and Sensitivity

Bias and sensitivity are key aspects that have been taken into consideration when identifying assessment items for the MPT.

Some aspects of bias and sensitivity were measured objectively during item field testing using parameters from item-response theory to identify geographic and other demographic indicators that predict reduced performance on items. These items were then further reviewed to identify more specific causes (e.g., religious holidays, celebrations, pets). Items in the item bank were carefully reviewed against a rubric that factors in identity, social justice, and equity issues prior to being released for field testing.

Non-Calculator Items and Calculator-Enabled Items

When a calculator is used to solve mathematics content problems, it is important for teacher applicants to know how to operate it properly. It is certainly a useful tool to work through problems while focusing on critical thinking. However, it is also important for teacher applicants to demonstrate the ability to perform mathematical operations without it.

To this end, the first 5 assessment items of the mathematics content component of the MPT are answered without the use of a calculator. Referred to as non-calculator items, these items are Number Sense items from the Knowledge and Understanding category of knowledge and skills and they test the skills of basic computations. Once teacher applicants complete these 5 items, they have access to the remaining 45 items that can be answered with the use (if desired) of an on-screen calculator available within the MPT platform. These items are referred to as calculator-enabled items.

Item Bank

Although each test form has 50 mathematics content items and 21 pedagogy items, the

item bank (test bank) consists of over 250 mathematics content items and 100

pedagogy items. This ensures a variety of items covering all subcategories and

supports the creation of unique testlets (blocks of items used to create the unique test

forms) of equal difficulty.

The item bank maintains critical metadata on each item. The item testlet allocation

metadata, for example, allows for an efficient pull of (pre-computed) psychometrically

equated test forms. The item bank also employs an immutable version control system of

items, to ensure that every edit made to an item is tracked.

of 27

Item Pre-testing

The initial 250+ items that make up the mathematics component of the MPT in Winter

2020 were pre-tested items that had been psychometrically validated. The initial

implementation of the MPT in Winter 2020 also acted as a pre-testing (field-test) of both

the mathematics content and pedagogy items, which have since been psychometrically

validated.

Item Field Testing

The mathematics assessment items for the MPT have strong psychometric properties to ensure a reliable and accurate reflection of the teacher applicants’ proficiency in mathematics. As much as the increase in the number of test sessions and test windows may affect a high exposure of items, which would motivate new item construction, the increased usage of the MPT also provides opportunities for field-testing newly constructed items. Field-test items are embedded in every MPT that is administered, to derive item statistics for equating purposes, and to ensure the continued high-quality standard of the item bank.

As described earlier, in every test administration, 4 field test items (that do not count towards the teacher applicant’s score on the test) are included in the MPT. These items are 1 non-calculator item, 2 calculator-enabled items, and 1 pedagogy item.

Questionnaire Items

In addition to the two components of the MPT, teacher applicants can choose to complete the Mathematics Attitude and Perceptions Survey (MAPS), a 32-item questionnaire designed is included to assess overall attitudes and dispositions towards mathematics at the undergraduate level. The survey was developed at the University of British Columbia (UBC), and its validation has been peer-reviewed and published (2016). As part of the survey, teacher applicants are asked to respond to identity-based questions to permit the examination of identity-informed outcomes.

of 27

Test Design

Test Design Criteria A test bank of assessment items (with psychometric properties) is available and

categorized into testlets (blocks) of items from which versions (test forms) of the MPT is

constructed.

Listed below are key criteria followed in the test design.5

✓ Each teacher applicant receives a unique test form.

✓ Each test forms are of equal difficulty.

✓ Each test form is equally representative of the assessment framework.

✓ Each test form has the same number of items.

✓ Each test form contains sufficient items for reporting purposes yet is short

enough to complete in a reasonable length of time.

✓ Each test form has enough items to enable reliable reporting of the performance

on both the mathematics content component and pedagogy component of the

MPT.

Assessment Delivery Method For a technology-based, modern, integrated assessment design, the MPT uses a

computer-based testing (CBT) method of delivery.

Today, the high level of sophistication that can be achieved with CBT includes:

• on-demand testing anywhere and at any time

• the ability to produce large numbers of equivalent test forms

• adaptive and multistage testing

• the use of technology-enhanced items and game-based items, and

• the ability to perform complex computerized exercises on low (to no) Wi-Fi

enabled devices

CBT delivery also creates numerous opportunities for assessing the performance and

cognitive ability of teacher applicants in many dimensions.

The MPT uses the testlet-based Linear-on-the-Fly Test (tLOFT) delivery method.6

The tLOFT is an assessment delivery method where test forms are generated in real

time by selecting pre-constructed testlets (blocks of items). This delivery method has

the following key features:

• It randomly chooses testlets that are categorized.7 • Every testlet is designed to meet the required content and statistical expectation.

5 The College Student Achievement Project Team (2015).

6 Numerous CBT delivery methods have been summarized and compared by Luecht & Sireci (2011); Folk & Smith (2002); Luecht (2013); and van der Linden (2006).

7 Wainer and Kiely (1987).

of 27

• It enables teacher applicants to review and change answers. • It provides a fixed test length. • It aligns to the multiple mathematics content dimensions and categories of

knowledge and skills.

Construction of Test Forms

The MPT has two sections: a mathematics content component and a pedagogy

component. The mathematics content component is divided into two sections: the first

one containing 5 items that are solved without the use of calculators (non-calculator

items) and the second one containing the remaining 45 items that may be solved with

the use of calculators (calculator-enabled items). The pedagogy component has 21

questions.

Mathematics Content Component

The item bank for the mathematics component is comprised of over 250 assessment items from across the mathematics content dimensions and categories of knowledge and skills. The items are categorized as Type A (Knowledge and Understanding), Type B (Application), and Type C (Thinking) items.

Out of the 250+ items, about 25 are non-calculator items that comprise the initial 5 questions of the test form. The remaining items (about 225) can be answered with the use of a calculator (if desired).

Selecting Non-Calculator Items from the Item Bank

Non-calculator items are those items for which the use of calculators is not permitted.

They test the skills of teacher applicants to perform mathematical operations.

The following example illustrates the selection of non-calculator items from the test bank

as shown with black borders and the creation of the test form.

With every attempt of the MPT, teacher applicants receive a unique test form. After

attempting the 5 non-calculator items, teacher applicants have access to the

45 calculator-enabled items.

5 items

KNOWLEDGE & UNDERSTANDING

NUMBER SENSE Covering all subcategories

of 27

Selecting Calculator-Enabled Items from the Item Bank

Based on the distribution, calculator-enabled items are selected from the item bank for

each test form.

The following example illustrates the selection of calculator-enabled items from the test

bank as shown with black borders.

The numbers of items selected from the test bank for each category of calculator-

enabled items are shown in the table below.

# MATHEMATICS CONTENT DIMENSIONS

CATEGORIES OF KNOWLEDGE AND SKILLS

NUMBER OF ITEMS

PER CATEGORY

PER DIMENSION

1 Number Sense Knowledge and Understanding 5 items 11 items

2 Number Sense Application 6 items

3 Relationships and Proportional Reasoning

Knowledge and Understanding 10 items

27 items 4 Relationships and Proportional Reasoning

Application 14 items

5 Relationships and Proportional Reasoning

Thinking 3 items

6 Measurement Application 5 items 7 items

7 Measurement Thinking 2 items

Total: 45 items

Covering all subcategories

MEASUREMENT

Covering all subcategories

RELATIONSHIPS &

PROPORTIONAL

REASONING

Covering all subcategories NUMBER SENSE

APPLICATION THINKING KNOWLEDGE & UNDERSTANDING

of 27

Testlets Construction

The following are key features in the construction of testlets of items for test forms:

• Each testlet of items has, as close as possible, the approximate difficulty level

(i.e., average p-value).

• Each testlet of items has a range of content tags (i.e., the range of

subcategories) that will be as broad as possible.

• Each number sense testlet has at least 1 type A item, at least 2 type B items, and

no type C items.

• Each relationships and proportional reasoning testlet has at least 1 type A item,

at least 2 type B items, and no more than 1 type C item.

• Each measurement testlet has no type A items, at least 1 type B item, and at

least 1 type C item.

The following example illustrates the selected calculator-enabled items used for the

construction of testlets.

Item Reordering and Test Form Creation

Once the testlets have been constructed, the calculator-enabled items are reordered,

and a unique test form is generated.

NUMBER SENSE RELATIONSHIPS & PROPORTIONAL

REASONING

MEASUREMENT

With every attempt of the MPT, teacher applicants receive a unique test form.

Each row of items represents a testlet.

of 27

Pedagogy Component

Pedagogy assessment items are sorted into the following three dimensions that make

up the MPT framework as per the distribution below: Growing Success, Learning for All,

and the Front Matter of The Ontario Mathematics Curriculum.

Selecting Items from the Item Bank

The item bank is comprised of over 100 assessment items from across the three

dimensions. Based on the distribution, items are selected from the item bank for each

test form.

The following example illustrates items that are selected from the test bank as

highlighted with a black border.

3 Items

GROWING SUCCESS LEARNING FOR ALL FRONT MATTER of the Mathematics Curriculum

9 Items 9 Items

of the Mathematics Curriculum

GROWING SUCCESS LEARNING FOR ALL FRONT MATTER

The number of items selected for each category is shown below.

The items are then reordered, and a unique test form is generated.

With every attempt of the MPT, teacher applicants receive a unique test form.

of 27

On-Demand Test Form Creation

When the teacher applicant is ready to begin the test, the testlets of items are

constructed automatically from the respective test banks to generate a unique test form

for each teacher applicant.

The first 5 questions to be solved without the use of a calculator are reordered and

placed at the beginning of the test. The 45 content questions that follow are reordered

and placed next, and finally, 21 pedagogy questions are selected, reordered and placed

at the end of the test form. This on-demand test form creation allows for numerous test

forms to be created for each test session and across multiple test sessions that equally

represent the MPT framework.

Reporting

When a teacher applicant completes the MPT, their test is automatically machine-

scored. Within 10 business days (after validation by EQAO), the teacher applicant

receives an email, generated from the MPT platform, with a report describing their

performance on both the mathematics content component and pedagogy component.

The report shows teacher applicants if they are “successful” or “not yet successful”

on the test. This is based on an achievement of at least 70% on each of the two

components: mathematics content component and pedagogy component. The report

shows their score (as a percentage) on each of the components.

Additionally, a visual representation is displayed for teacher applicants to view a

breakdown of their performance on the three mathematics content dimensions and

knowledge and skills categories for the mathematics content component, and the three

dimensions of the pedagogy component.

Their performance is indicated with two colours:

If teacher applicants are not successful on the MPT:

• They can use their performance feedback provided on the report to improve their

skills and reattempt the test.

• They are required to reattempt the entire test (both components), irrespective of

their results on individual components on previous attempts.

• They can reattempt the test based on test session availability at the test centre.

Green (G) represents a score above the threshold.

Amber (A) represents a score under the threshold.

of 27

An example of a report that teacher applicants receive is shown below.

of 27

Accommodation and Accessibility

Accommodation

Accommodation supports learning and aids in the expression of understanding of

concepts and ideas.

Test administrators can provide appropriate accommodation to teacher applicants

without compromising the demonstration of knowledge and skills reflected in the

components of the MPT. According to the Ontario Human Rights Commission, it is

necessary for test providers to “Implement policies and procedures to ensure that

students with disabilities receive appropriate, dignified, and confidential

accommodations to testing procedures” (Ontario Human Rights Commission, 2003,

p.78).

Some ways in which test centres can provide accommodation for teacher applicants

with respect to the MPT are listed below:

● additional and unlimited breaks throughout test sessions

● distraction reduced areas for test sessions

● extension of allocated time

● use of a reader for test sessions

● use of a scribe for test sessions

Accessibility

The MPT platform accommodates teacher applicants with accessibility requirements, up

to the standard of Web Content Accessibility Guidelines (WCAG) 2.0 Level AA. This is

the requirement set by the Ontario government for public websites to fulfil by January 1,

2021, in order to comply with the Accessibility for Ontarians with Disabilities Act

(AODA). The purpose of these standards is to provide alternative content or

functionality for teacher applicants with different learning needs, including:

• Blindness and low vision

o Screen zooming, high contrast mode, text alternatives for colour-

dependent items, compatibility with screen reader software

• Deafness and hearing loss

o Design that avoids the need for deafness and hearing loss

accommodation

• Mobility impairments

o Alternatives to drag and drop items

o Full keyboard navigation

• Speech impairments

o Design that avoids a need for speech accommodation.

The MPT platform is compliant with the principles of the WCAG, in that it implements

content that is Perceivable, Operable, Understandable, and Robust for all users.

of 27

Data Analysis and Security

Data Analysis

Teacher applicant response data and activity data from the MPT platform enables

insights primarily into performance on test content and into general use of the platform.

Of key interest is how teacher applicants respond to items with respect to both the

range of responses (including whether they attempted an item) and their performance

on the test overall as well as on an item-by-item basis. Some further insight into test

difficulty is provided with response-level timestamps and time spent.

Analysis on the overall use of the platform includes how often teacher applicants are

interacting with the platform, which test windows are frequented, and how many test

sessions have closed or have been completed.

Field testing was accompanied by psychometric analysis to establish test reliability. The

results from this process ensure that data analysis carried out by EQAO produces valid

and actionable results.

Security

Test security is a key concern regarding the MPT. Measures have been implemented to

guard against compromised test questions and/or answers. Using a large pool of items

to construct test forms provides assurance that each test form will be unique. The large

test bank ensures that a possible leak would not be representative of the test bank.

Another key element concerning the security of the MPT is how data is governed and

who has the right to access the data collected. An MPT Data Governance, established

by the EQAO, ensures that all stakeholders understand their role in the data processing,

and more importantly, that restrictions on data processing protect the freedoms and

privacy rights of the teacher applicants.

The MPT follows the high standards set by the Government of Ontario Information and

Technology Standards (GO-ITS) for Cloud Services 25.21. The adoption of these

standards by the Government of Ontario sets a high benchmark regarding guidelines,

technical reports, and preferred practices. In line with the GO-ITS, the MPT platform is

subject to the standard audits, documentation, and policies recommended to ensure

that the platform follows high standards of security.

of 27

References Code, W., Merchant, S., Maciejewski, W., Thomas, M., & Lo, J. (2016). The

mathematics attitudes and perceptions survey: An instrument to assess expert-like

views and dispositions among undergraduate mathematics students. International

Journal of Mathematical Education in Science and Technology, 47(6), 917-937.

http://dx.doi.org/10.1080/0020739X.2015.1133854

Education Quality and Accountability Office (2019). Highlights of the provincial results:

Mathematics. Toronto: Queen’s Printer for Ontario.

Education Quality and Accountability Office (2019). Math trends in Ontario at a glance.

Retrieved from http://www.eqao.com/en/about_eqao/media_room/communication-

docs/infographic-math-trends-ontario.pdf

Education Quality and Accountability Office (2019). Questions and answers for those

writing the new Mathematics Proficiency Test. Retrieved from:

http://www.eqao.com/en/assessments/math-proficiency-test/Pages/questions-

answers-math-proficiency-test.aspx

Folk, V. G., & Smith, R. L. (2012). Models for delivery of CBTs. In C. Mills, M Potenza,

J. Fremer, & W. Wards (Eds), Computer-based testing: Building the foundation for

future assessments (pp. 41-66). Mahwah, New Jersey: Lawrence Erlbaum.

Leucht, R. M. (2013). An introduction to assessment engineering for automatic item

generation. In M.G. Editor & T. H. Editor (Eds), Automatic item generation: Theory

and practice. (p.18). New York, New York: Routledge.

Luecht, R. M., & Sireci, S. G. (2011). A review of models for computer-based testing:

Research report 2001-12. New York, New York: College Board.

Martin, T.S., & Herrera, T. (2010). Standards for teaching and learning mathematics.

Mathematics teaching today: Improving practice, improving student learning

(pp. 1-26). Michigan: National Council of Teachers of Mathematics.

Office of the Auditor General of Ontario (2009). Education Quality and Accountability

Office. 2009 Annual Report: Office of the Auditor General of Ontario (128-142).

Toronto: Queen’s Printer for Ontario.

Ontario Human Rights Commission (2003). The opportunity to succeed: Achieving

barrier free education for students with disabilities. Toronto: Queen’s Printer for

Ontario.

Ontario Ministry of Education. (2005a). The Ontario Curriculum, Grades 1-8:


Ontario Ministry of Education. (2005b). The Ontario Curriculum, Grades 9-10:


of 27

Ontario Ministry of Education (2010). Growing success: Assessment, evaluation, and

reporting in Ontario Schools, first edition, covering Grades 1 to 12. Toronto: Queen’s

Printer for Ontario.

Ontario Ministry of Education (2013). Learning for all: A guide to effective assessment

and instruction for all students, Kindergarten to Grade 12. Toronto: Queen’s Printer

for Ontario.

Ontario Ministry of Education. (2016a). The Ontario Curriculum, Grades 9-12: Financial

Literacy - Scope and Sequence of Expectation. Toronto: Queen’s Printer for Ontario.

Ontario Ministry of Education. (2016b). The Ontario Curriculum, Grades 4-8: Financial

Literacy - scope and sequence of expectation. Toronto: Queen’s Printer for Ontario.

O. Reg 271/19: Proficiency in Mathematics under Ontario College of Teachers Act, 1996, S.O. 1996, c. 12. Retrieved from https://www.ontario.ca/laws/regulation/190271

Orpwood, G., & Brown, E. (2015). Closing the numeracy gap: An urgent assignment for

Ontario. Retrieved from

http://www.numeracygap.ca/assets/img/Closing_the_numeracy_gap_V4.pdf

Randall, I.C. (2005). Big ideas and understandings as the foundation for elementary and

middle school mathematics. Journal of Mathematics Education Leadership,7(3), 9-24.

The College Student Achievement Project Team (2015). College student achievement

project: Final report 2015.

Van der Linden, W.J. (2006). A lognormal model for response times on test

items. Journal of Educational and Behavioral Statistics, 31, 181–204.

Wainer, H., & Kiely, G.L. (1987). Item clusters and computerized adaptive testing: A

case for testlets. Journal of Educational Measurement, 24, 185-201.


the ontario mathematics proficiency test (mpt)

Documents