community participation in measuring learning outcomes in
TRANSCRIPT
Community Participation in Measuring Learning Outcomes in Remote Areas of IndonesiaResults from the Development and Implementation of Tes Cepat Teacher Performance and Accountability (KIAT Guru)
BACKGROUND PAPER
MARCH 2021
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
This work is a product of the staff of The World Bank. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries.
Rights and Permissions© 2021 The World Bank 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org
Some rights reserved.
The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for non-commercial purposes as long as full attribution to this work is given.
All queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: [email protected].
AttributionPlease cite the work as follows: Lumbanraja, S. K.; Prameswari, I.A.; Susanti, D. 2021. Community Participation in Measuring Learning Outcomes in Remote Areas of Indonesia Results from the Development and Implementation of Tes Cepat Teacher Performance and Accountability (KIAT Guru) - Background Paper, World Bank, Jakarta. © World Bank.
Photo credits Fauzan Ijazah/World Bank.
BACKGROUND PAPER
Community Participation in Measuring Learning Outcomes in Remote Areas of Indonesia
Results from the Development and Implementation of Tes Cepat
Teacher Performance and Accountability (KIAT Guru)
SOCIAL SUSTAINABILITY AND INCLUSION UNIT
THE WORLD BANK – INDONESIA
2021
Table of Contents
List of Abbreviations v
01 Introduction
02 Test Design
03 Test Implementation
1
3
11
Principles 4
Citizen-Led 4
Foundational Skills 5
Adaptive 5
Results for Actions 5
Framework 7
Initial Framework 7
Expanded Framework 8
Test Format and Administration 8
Test Format 8
Test Administration 9
Test Scoring and Dissemination 9
Test Scoring 9
Test Dissemination 9
Training 12
Recruitment 12
Methods 12
Administration 13
Actions from Results 15
CONTENTS
i i .
04 Conclusion 17
Summary 17
Lessons from Paper and Digital Tes Cepat 18
Future Development 19
References 21
Appendix 1. Reading and Math Competencies Measured in Tes Cepat Initial Framework 23
Appendix 2. Reading and Math Competencies Measured in Tes Cepat Expanded Framework 24
CONTENTS
i i i .
FiguresFigure 1 Four Principles of Tes Cepat Design 4
Figure 2 Parents’ Education Level in KIAT Guru Schools 4
Figure 3. Steps in Implementing Tes Cepat Adaptive Flow 5
Figure 4. Example of Primary Grade 1 and 6 Reading Tasks in Tes Cepat 7
Figure 5. Example of Primary Grade 1 and 6 Math Tasks in Tes Cepat 8
Figure 6. Sample Map of Student Ability Against the National Curriculum as Measured by Tes Cepat 9
Figure 7. Steps in Training, Administering, and Disseminating Tes Cepat 11
Figure 8. Steps and Methods in Training Tes Cepat to Community Members 12
Figure 9. Handbook in Administering Adaptive Testing on Reading Items in Paper Tes Cepat 14
Figure 10. Digital Tes Cepat 15
Figure 11. Paper Format Tes Cepat 15
Figure 12. Template to Map Student Ability from Paper Tes Cepat Results 15
Figure 13. Automated Map of Student Ability from Digital Tes Cepat 16
TablesTable 1 Differences between previous Citizen-Led
Assessments and Tes Cepat 6
Table 2 Rubric to Score Tes Cepat Item to Its Grade-level Equivalence 9
Table 3 Service Agreement to Increase Reading and Math Learning in School and at Home based on Tes Cepat Results 16
Table 4 Differing Advantages and Disadvantages in Implementing Paper and Digital Tes Cepat 18
CONTENTS
iv .
List of Abbreviations
ABBREVIAT IONS
v .
ACDP Analytical and Capacity Development Partnership
CREDO Creative Education Indonesia Foundation
ASER Annual Status of Education Report
KIAT Guru Teacher Performance and Accountability (Kinerja dan Akuntabiltas Guru)
KTSP 2006 Curriculum (Kurikulum Tingkat Satuan Pendidikan)
MoEC Ministry of Education and Culture
NTT Nusa Tenggara Province (Nusa Tenggara Timur)
OECD Organisation for Economic Co-operation and Development
PAL Network People’s Action for Learning Network, a south-south partnership of organisations working across three continents.
PUSPENDIK Education Assessment Centre (Pusat Penilaian Pendidikan)
SD Primary school (Sekolah Dasar)
SLA Student Learning Assessment
Tes Cepat Adaptive student learning assessment to identify and map student functional literacy and numeracy skills along a continuum
TNP2K The National Team for the Acceleration of Poverty Reduction (Tim Nasional Percepatan Penanggulangan Kemiskinan)
UN National Examination (Ujian Nasional)
v i .
The Indonesia KIAT Guru: Improving Teacher Performance and Accountability analytical study was led by Dewi Susanti. Sharon Kanthy Lumbanraja and Indah Ayu Prameswari wrote this technical paper. A large number of people contributed to the design, data collection and data analysis of the Tes Cepat. For design, we are grateful to Dewi Susanti, Gunawan, Marliyanti, Setiawan Cahyo Nugroho, Ranajit Bhattacharyya, Rukmini Banerji, and the People’s Action for Learning (PAL) Network. We also thank Arya Gaduh, Jan Priebe, and Menno Pradhan for guidance and assistance with analytical works. Data collection was implemented by Yayasan BaKTI, under the leaderships of Caroline Tupamahu, Setiawan Cahyo Nugroho, and Tri Yuni Rinawati at the national level, and Denta Romauli Sihombing, Rustanty Dewi, Handaru Suryo Putro, Nugroho, and Safrijal at the district levels. Zenius developed the digital Tes Cepat, led by Rizky Andriawan and Zulfikar Hermawan. Chatarina Ayu Widiarti, Dinda Putri Hapsari, Fazlania Zain and Rebekka Hutabarat provided overall support for the implementation and publication. Purwa Rahmanto provided graphic design and formatting support for the publication.
The World Bank is thankful to the Indonesian Ministry of Education and Culture (MoEC) and the National Team for Acceleration of Poverty Reduction under the Office of the Vice President of Indonesia (TNP2K) for their advice and support. The World Bank is especially thankful to Bapak Bambang Widianto, Bapak Nizam, and Ibu Rahmawati for their insightful guidance and support. The Government of Australia’s Department of Foreign Affairs and Trade (DFAT) and United States Agency for International Development (USAID) generously financed this study. Research for Improving Systems in Education (RISE) Indonesia, managed by SMERU Research Institute, co-financed the second round of surveys. In addition, the task team is grateful for the guidance extended by the Country Management Unit of the World Bank in Indonesia. Kevin Tomlinson, Nina Bhatt, and Janmejay Singh provided oversight of the task team.
Acknowledgments
1.
Introduction
INTRODUCTION
01Learning outcomes of students in remote areas of Indonesia are behind their urban peers (ACDP, 2014; Stern & Nordstrum, 2014). KIAT Guru pilot aims to improve teacher performance and accountability in Indonesia’s remote areas by means of community empowerment and performance-based teacher pay (Gaduh et al, 2020). A 2016 survey conducted by the World Bank in KIAT Guru’s schools indicated that 25 percent teachers were missing from classrooms, and most students were performing two grade levels below their current grades. Surprisingly, 83% of the students’ parents were found to be satisfied with the schools (World Bank, 2019).
KIAT Guru empowers community by increasing their demand for quality education. Parents and teachers developed a Community Score Card (adapted from Bjorkman & Svensson, 2009), which would be monitored every month. It contained set of service indicators for both school and community members to improve quality of education service in their village. These indicators were developed and evaluated based on evidence, including basic literacy and numeracy outcomes.
Parental involvement has been found to be a particularly critical factor in improving learning outcomes in rural areas of developing countries. Initiatives such as Citizen-led Assessment, dissemination of learning information, and community-based monitoring on teacher accountability aim to tackle disparity in student achievement living in disadvantaged areas (ASER, 2014; Dizon-Ross, 2016; Barr et al., 2012). Effects of community monitoring on service delivery improvements have been encouraging (Banerjee & Duflo, 2006; Joshi, 2010; Ringold et al, 2012). However, this does not discount the importance of teacher competency in providing engaging learning environment towards student achievement (Fazih et al., 2018).
Parents have limited awareness on student learning outcomes. In most Indonesian schools, parents are usually informed about their children’s learning progress through report cards on a semester basis. Additionally, scores marked in these reports do not necessarily provide concrete insights to individual students’ strength and weakness for each subject matter. As these scores are given by teachers, they tend to be subjective, and thus difficult to compare with scores from other schools or even other grades within the same school. The national exam on the other hand is only catered to measure sixth grade student performance across the country. Similarly, results from this school-proctored exam are also often difficult for parents to decipher.
INTRODUCTION
2.
This background paper documents the development, implementation, and results from Tes Cepat that support community participation to advocate for improvements in student learning outcomes. It also highlights the recent digitization of the test into a mobile phone application. The paper is organized as follows: Section 2 discusses the design of the test; Section 3 describes the implementation of the test; and Section 4 concludes.
KIAT Guru developed Tes Cepat, a low-stake diagnostic test on primary grade reading and math administered by community members on a semester basis. Tes Cepat is modeled after the Citizen-led Assessment movements by Pratham ASER (2014) and Peoples Action for Learning (PAL) Network (2017). The test maps students’ results in reading and math against the national curriculum standard. Results are then informed to and used by teachers, parents and community members to develop and evaluate service indicators in the Community Score Card. This information is found simple enough for parents and teachers to understand and act upon to improve their children’s learning.
3.
Test Design02
Tes Cepat was designed with four main principles: citizen-led, foundational skills, adaptive to student’s ability, and oriented on results for actions (Figure 1). It was modeled after Pratham’s Annual Status of Education Report (ASER)1 and other Citizen-Led Assessments (CLA) implemented in the PAL Network2 countries. The next subsections explain in greater detail how each principle is designed in Tes Cepat.
Tes Cepat is designed based on four main principles: citizen-led, foundational skills, adaptive to student’s ability, and oriented on results for actions. Tes Cepat is made simple to administer and easy to understand by citizens. Results from the test are then used to mobilize citizens, teachers, and other relevant education stakeholders to take actions in improving the quality of local education service delivery. Tes Cepat framework was initially adapted from Student Learning Assessment (SLA) tool, which adheres to both national (2006) and international standards (TIMSS, PIRLS). It has recently been expanded to align with the latest national (2013) and international standards (PISA). Tes Cepat was initially developed as a paper test, and in 2019 it was transformed into a digital mobile phone app. In both formats, test items are written in a multiple-choice format. The test is administered individually, with instructions given verbally by administrator, or combined with the app for the digital format. Tes Cepat is then scored based on the most difficult item student performs correctly on, which is translated to its grade level equivalence. Results are displayed by mapping the distribution of student ability against national curriculum standards. This learning information is disseminated in a forum attended by citizens, teachers, village government, and other education stakeholders. In KIAT Guru schools, this information is used as a reference to develop a service agreement for community and school to collaborate in improving quality of learning in school and at home.
1 Annual Status of Education Report or ASER is a national citizen-led rapid assessment of children’s ability to read simple text and do basic arithmetic. ASER is designed and facilitated by the Indian nongovernment organization Pratham, and has been conducted every year since 2005 by partner organizations in every rural district of India. 2 The People’s Action for Learning Network (PAL Network) is a south-south partnership of organizations working across three continents. Member organizations conduct citizen-led assessments and/or citizen-led actions aimed at improving learning outcomes. https://palnetwork.org/
TEST DESIGN
TEST DESIGN
4.
2.1 Principles
2.1.1 Citizen-Led
Tes Cepat was designed as to increase community awareness of learning crisis by engaging them in the assessment process. Improved transparency and access to information are key to social accountability (Read & Attinc, 2017). To increase citizen participation in education specifically, they must first realize of such learning crisis. This awareness comes with direct involvement in measuring learning outcomes. It will allow citizens to witness firsthand the struggles and advances in student learning. Providing such information is found to be a sufficient and inexpensive way to increase community involvement in education (Banerjee et al., 2010). Similar to cases in India and other countries of the PAL Network, citizen led assessments however have not been a commodity that is widely available or easily accessible in Indonesia.
Citizen involvement ensures objectivity of test results. As beneficiaries of education services, parents and community members have the genuine interest to be informed on the actual state of their children’s learning. Such motivation enables them to be impartial data collectors, making results of the test more objective. This is in contrary to the risk of having education service providers in conducting the test. Specifically, teachers will be more reluctant to reveal any failures in the services that they have provided (Read & Attinc, 2017).
Figure 1. Four Principles of Tes Cepat Design
Tes CepatCitizen-Led Foundational Skills Adaptive Results for Actions
Easy to Administer by Local Citizens
Simple to Understand for Citizens and Students
Actionable Results for Citizens and Teachers
Parents and community members are primary actors of Tes Cepat as it is easy to administer and simple to understand for citizens of any education background. Unlike its CLA predecessors, Tes Cepat is mainly implemented by local community members. Understanding the test not only increases their awareness, but also builds local community capacity in monitoring educational outcomes. As 60 percent parents (figure 2) in KIAT Guru schools have primary grade level education, instructions to administer and score Tes Cepat were designed to be simple and easy to follow. It does not however discount the breadth and quality of learning information that the tool yields.
Figure 2. Parents’ Education Level in KIAT Guru Schools
No education
Primary school
Junior secondary school
Senior secondary school
University
60%19%
13%
2%6%
TEST DESIGN
5.
2.1.2 Foundational Skills
Tes Cepat measures basic reading and math skills. As foundational skills to learn (OECD, 2019), information on primary grade reading and math learning outcomes serves as a powerful indicator to the quality of basic education services. Tes Cepat is designed to capture student’s ability on grade 1 through 6 reading and math standards as set by the national curriculum.
2.1.3 Adaptive
Tes Cepat is adaptive to student ability. As a simple tool to administer, the test is designed to be quick to implement while also comfortable for students to work on. Adaptive test presents items based on student’s performance. Specifically, students will be given harder or easier test items following a correct or incorrect answer, respectively. This approach prevents students from being intimidated and enables them to perform
according to their actual ability. Figure 3 presents the steps in implementing Tes Cepat adaptive flow.
2.1.4 Results for Actions
Tes Cepat yields low-stake yet actionable results for citizens and teachers. In the spirit of other CLAs, Tes Cepat was not developed simply to collect learning data for its own sake. As reported by Read and Attinc (2017), information itself is not the core of CLAs, it is what the information captured, and how it is shared that matters. Tes Cepat was developed with the intent that every child should be able to read and be numerate. Results from Tes Cepat should lead to action. Results are designed to be comprehensible for all education stakeholders, including citizens and teachers. In the context of KIAT Guru, learning information yielded by Tes Cepat is used to help parents and teachers to develop joint service between what teachers could do in school and what parents could follow up at home to improve learning.
Figure 3. Steps in Implementing Tes Cepat Adaptive Flow
Example:
Tono, a former 5th grader in 2016/2017 academic year will be given item #30 on page 13 as the first item on the last page for fifth grade in the Math Test Booklet.
KELAS 5 TERAKHIR
30. Perhatikan gambar di bawah. Sisi yang sama luas dengan ABCD adalah ...
31. Pilih gambar yang bukan merupakan jaring-jaring kubus adalah ...
31. Murid kelas 5 di SD Pagi Sore ada 40 anak. Perbandingan murid perempuan dengan semua murid di kelas 5 adalah 5 : 8. Ada berapa banyak murid perempuan di kelas 5 di SD Pagi Sore?
A B
D C
E F
H G
ADHEa BCGFb EFGHc EFCDd
a b c d
5a 25b 8c 13d
HALAMAN 13
STEP 1 STEP 2 STEP 3 STEP 4 END OF TEST
Identify student’s
grade level based on
2016/2017 Academic Year
(Baseline)
First item on the last page consisting of
items at child’s grade level
Can the child answer correctly?
First item on the last page consisting of
items at child’s grade level
Can the child answer correctly?
Continue to the next (higher number) item
Can the child answer correctly?
First item on the last page for one grade level below
Skip further down until child is able
to answer correctly
Continue to the next (higher
number) items
Until child answers
incorrectly
Mark child’s level based on
the last (highest) test
item answered correctly
When child stop
STOP
Continue to the next (higher
number) items
Until child answers
incorrectly
NO
NO
NO
YES
YES
YES
TEST DESIGN
6.
Tes Cepat serves as a regular monitoring tool for citizens and teachers to measure learning outcomes. Citizens participate more effectively when they can regularly monitor the outcomes of the local education services. As a formative assessment, Tes Cepat provides semesterly learning information that consequently stimulates objective discussion between citizens and teachers on ways to collaborate in improving education service delivery. Providing actionable information that can be monitored overtime can foster more involvement (Banerjee et al, 2010).
Overall, Tes Cepat adopts CLA main principles that builds the capacity of citizens to measure, understand, communicate, and act upon the results of the test, with the objective of improving the quality of education services. Quoting from a report on ASER, “… [citizen led assessment] aims to create a culture where rigorous measurement of outcomes is integral to action, thus bridging the gap between theory and practice, assumption and reality (n.d.).” Several features of Tes Cepat are however expanded from CLA original forms. Table 1 presents how Tes Cepat differs from its CLA predecessors.
Table 1. Differences between previous Citizen-Led Assessments and Tes Cepat
Feature Other CLAs: ASER, Uwezo3, MIA4, etc. Tes Cepat
Framework Foundational literacy and numeracy skills, equivalent to primary grade 1-3 standards
Foundational literacy and numeracy skills, equivalent to primary grade 1-6 standards
Sample In- and out- school children age 6-14 in urban slums and remote villages
In-school children grade 1-6 in remote villages
Administrator Volunteers from partner NGOs
Local village community members (initially paired with NGO facilitators)
Action Teaching at the Right Level (TaRL)5 and Reading Camps conducted by community volunteers and teachers
Service Agreement between Teachers and Parents
3 Uwezo conducts annual large scale, citizen-led, household-based assessments that measure actual levels of children’s literacy and numeracy across Kenya, Tanzania and Uganda. https://www.uwezo.net/about-us/our-goal/4 Medición Independiente de Aprendizajes (MIA), or independent measurement of learning, is the first citizen-led assessment in Latin America. The mission of MIA is to increase the participation of civil society in education through collaborative research involving citizen volunteers; local civil society organizations, universities and the education community. http://www.medicionmia.org.mx/ 5 Teaching at the Right Level (TaRL) is an approach pioneered by Indian NGO Pratham. The approach works by dividing children (generally in Grades 3 to 5) into groups based on learning needs rather than age or grade; dedicating time to basic skills rather than focusing solely on the curriculum; and regularly assessing student performance, rather than relying only on end-of-year examinations. https://www.teachingattherightlevel.org/
TEST DESIGN
7.
2.2 Framework
Tes Cepat framework adheres to latest developments in national and international standards on basic reading and math. Tes Cepat framework to measure reading and math is primarily adapted and modified from KIAT Guru’s Student Learning Assessment (SLA). The SLA was developed as a survey tool to evaluate impact of the pilot interventions on learning outcomes (Lumbanraja & Prameswari, forthcoming). The framework aligns to both national and international standards on primary grade reading and math. Built on its second principle however, Tes Cepat only focuses on fundamental skills in these two domains. Since the test inception in 2016, Tes Cepat has gone through iterations to adjust with the latest development with national curriculum. The following subsections present the test initial and later revised frameworks.
2.2.1 Initial Framework
Tes Cepat was initially built upon the 2006 national curriculum as well as the PIRLS and TIMSS standards. It was the simplified version of the SLA. Test items were developed with assistance from curriculum
experts and MoEC’s Center for Education Assessment (Pusat Penilaian Pendidikan/ Puspendik).
Using 1 or 2-PL IRT analysis, Tes Cepat test items had sound item statistics. The psychometric quality of the SLA fulfills criteria set by Puspendik (2012) following rounds of test piloting. In its simplification process, a total of six to eight test items per grade level, or three percent of the SLA, selected to be included in the Tes Cepat. Although much fewer in quantity, these test items had strong statistical results and represent primary content domains covered.
Reading and math items relevant to children’s functional skills are assessed. Similar to ASER and other CLAs (Pratham, n.d.; PAL Network, 2018), skills measured in Tes Cepat serves as an important building block for children to function within the education system and life in general. It is also practical for citizens and teachers to discuss and act upon learning information on these skills. Appendix 1 further lists reading and math competencies included in Tes Cepat items along with their grade-level equivalence as set by the 2006 national curriculum. Figure 4 and 5 are samples of reading and math functional skills assessed in Tes Cepat.
Figure 4. Example of Primary Grade 1 and 6 Reading Tasks in Tes Cepat
Pada suatu hari ketika masih SD, saya menemukan kucing tergeletak di jalan setapak. Kucing itu kakinya patah karena tertabrak mobil. Setelah saya obati dan saya rawat, akhirnya kucing itu sembuh. Kini saya bekerja mengobati dan merawat orang sakit.
37 Saya dalam bacaan di atas adalah ...
a Dokter
b Guru
c Petani
d Pedagang
Letter Recognition Reading Comprehension: Drama Text
1. y t r w
2. q d b i
TEST DESIGN
8.
2.1.1 Expanded Framework
Tes Cepat framework is broadened to include latest national curriculum standards and higher-order thinking skills in international testing. In expanding its item bank, Tes Cepat is also revising the competencies it measures. Table 3 presents list of additional reading and math competencies that the test measures. These competencies were selected to align to the latest 2013 national curriculum standards, as well as to more higher-order thinking tasks that are core to 2018 PISA framework (OECD, 2019).
2.2 Test Format and Administration
2.2.1 Test Format
As a simple tool to administer, Tes Cepat test items were written as multiple-choice questions with three to four answer options. The answer options,
Figure 5. Example of Primary Grade 1 and 6 Math Tasks in Tes Cepat
38. Sari adalah seorang murid Kelas 6 di SD Pondok Baru. Hari ini Sari mendapatkan rapornya. Berikut nilai rapor Sari.
NO MATA PELAJARAN NILAI
1 Matematika 9
2 Bahasa Indonesia 8
3 IPA 6
4 IPS 8
5 Agama 9
6 Pendidikan Jasmani, Olahraga dan Kesehatan 8
7 Pendidikan Kewarganegaraan 7
Jumlah 56
Berapa nilai rata-rata yang didapatkan Sari?
a 7 b 5,6 c 7,5 d 8
Number Recognition Data & Statistics
1. 13 10 16 19
2. 96 59 86 69
both correct and incorrect, were constructed carefully to reduce any external factors aside to student’s ability. For reading items, these included random ordering of the correct option across questions and applying similar length of sentence or paragraph across options. For math items, these included having similar numbers across options, and using results from applying incorrect concept or procedure in solving the problem.
Initially developed as a paper-based test, Tes Cepat has recently been transformed into a digital format to increase its efficiency and ease of use. As a paper-based test in 2016, Tes Cepat was available in the form of test booklets, one for each subject. In 2019, it was digitized and is now available as an Android-based mobile phone application. This app is currently free for public to download through Google Play Store. The application was developed to increase efficiency and accuracy of the test development, administration, and scoring.
3 An example would be a subtraction problem with an answer option that applies addition instead of subtraction to the numbers presented in the problem. Another example would be a geometry problem that asks for an area of a shape, but instead provides an answer option that calculates the perimeter of the shape.
TEST DESIGN
9.
2.2.2 Test Administration
To ensure students’ ease of understanding, Tes Cepat is administered individually with verbal instructions. This allows the test to be understood even by students who are unable to read or write. Each student receives verbal instruction from the administrator throughout the testing session. To prevent administrators from aiding students with the test, they are given a separate set of prompts or instructions that can be read aloud to students. For paper based Tes Cepat, students provide their answer verbally, through writing, or by pointing the option they choose, which is then recorded by the administrator. This way, the test booklets can be used repeatedly for other students. For digital format, students provide their answer by clicking on the screen the answer option of their choice.
Parents and community members who administer the test are required to build rapport with students. Making students comfortable throughout the test is key to ensuring validity of the test results. It is imperative for students to experience a relaxed and encouraging environment to complete the test. Administrators are also required to speak slowly and clearly to ensure that all children can fully understand the expectation from the task.
2.3 Test Scoring & Dissemination
2.3.1 Test Scoring
Tes Cepat results identify where students are in the primary grade learning continuum. This allows both citizens and teachers to understand whether their children are reading or doing math according to their grade-level standards. Students’ performance in Tes Cepat is scored by the most difficult item that they can answer correctly. This item is then mapped into its grade-level equivalence based on the national curriculum standard. Table 2 is a rubric to map each item in the paper-based Tes Cepat to its grade-level equivalence. The same rule applies to digital Tes Cepat. The app automatically maps the last item that student answers correctly without having to display its number for the administrator to score.
Table 2. Rubric to Score Tes Cepat Item to Its Grade-level Equivalence
Competency Reading/ Math Item #
Illiterate/Innumerate 0-1
Below Basic 2-7
Grade 1 8-11
Grade 2 12-17
Grade 3 18-23
Grade 4 24-29
Grade 5 30-35
Grade 6 36-38
2.3.1 Test Dissemination
Distribution of student ability is displayed on a visual map that can be understood by both citizens and teachers. Lessons from ASER and other CLAs highlight that “…dissemination tools of learning information are as important as the source of the data itself” (Read & Atinc, 2017). In Tes Cepat, a grid-map is used to visualize distribution of student ability from each grade level as compared to the national curriculum standard (Figure 6). Such form allows learning information from the test to be understood quickly by both teachers and parents. Identity of tested students are not disclosed in this map as they represent the ability of their grade level group as whole.
Figure 6. Sample Map of Student Ability against the National Curriculum as measured by Tes Cepat
TEST DESIGN
10.
Learning information is disseminated at a village forum attended by all relevant education stakeholders. Tes Cepat results are expected to bring actions. In education, powerful actions can only occur when all relevant actors in children’s learning are participating. Building on Uwezo’s experience (Carlitz & Ripovsek, 2017), dissemination of learning information from CLAs should target relevant and capable stakeholders in education, including teachers, local leaders, and community-based organizations. This gives more power for citizens to act and improve education quality for their children. Information from Tes Cepat should always be disseminated in a forum attended by teachers, parents, community members, and village government officials. In the context of KIAT Guru, this information serves as a reference to develop service agreement that both school and community are responsible for as means to collaborate in improving the learning outcomes of their children.
TEST IMPLEMENTATION
11.
Tes Cepat was implemented based on a set of steps and guidelines in its training, administration, and actions from results. Figure 7 outlines the steps required to ensure quality field implementation of Tes Cepat, including use of its results to create actions by citizens and teachers. The following subsections describe in greater detail experience gained in implementing Tes Cepat across KIAT Guru pilot schools.
Figure 7. Steps in Training, Administering, and Disseminating Tes Cepat
Test Implementation03Tes Cepat followed a set of steps in conducting its training, administration, and actions from results. Citizens who are recruited and trained as administrators must have basic literacy skills, and ability to operate mobile phones. For time and cost efficiency, trainings usually occur at the village level. Both lectures and role plays are necessary methods to ensure citizens understand why the test is needed, and how it is conducted. Tes Cepat is implemented to a sample of 36 students per school and can be administered at school or at home. The test takes about 15 minutes per student as it is administered adaptively. Student results are then converted into grade-level abilities and plotted into a visual map. Tes Cepat results are disseminated by citizens and teachers at a forum attended by other community members and village government officials. Service agreement is then developed by community and teachers, which contains a set of indicators for both stakeholders to do to improve learning environment in school and at home.
Training of Trainers
Training of Community Members
Location, Time,
Sample
Test Scoring
Test Dissemination
Adaptive Administration
1 2 3 4 5 6
Training Administration Actions from Results
TEST IMPLEMENTATION
12.
3.1 Training
3.1.1 Recruitment
Test administrators must be able to read, write, and operate mobile phones. Because Tes Cepat’s instructions are required to be read aloud to students, administrators must be able to read these prompts. For paper Tes Cepat, administrators are also required to write down student’s information and results. For digital Tes Cepat, administrators must also be able to operate mobile phones. These basic skills can however be challenging to find especially in community members of remote villages in Indonesia.
3.1.2 Methods
Trainings are most effective when done at the village level and is closely assisted by KIAT Guru trainers or program facilitators. Not only was it cost efficient, community members were also more flexible with time when training activities took place within the village. This is imperative because community members varied in their ability to understand and their availability during the day, given they were not financially rewarded for participating in the training. Training locations also varied according to resources and time. Some were done inside the school’s classrooms, and some took place at the house of a local villager. Following the training, community members were able to easily coordinate with teachers on scheduling the test. If the test was scheduled right after the training, trainers could also spend an additional day or two to assist community volunteers with conducting the test.
A combination of lecture and role plays is necessary for community members to understand the Tes Cepat. Figure 8 maps out the three main contents of the training, which aim to address why the test is important, how the test is conducted, and what the test results are used for. These contents are presented through a combination of lectures and role plays to ensure community volunteers understood by listening as well as by doing.
Community administrators are recruited on a voluntary basis. In the period of
2016-2019, a total of
839 were identified by KIAT Guru program
facilitators or by village leaders. KIAT Guru program facilitators trained them, and they
administered Tes Cepat in
410 remote schools in Indonesia.
Figure 8. Steps and Methods in Training Tes Cepat to Community Members
Background and Objective of Assessment
Sampling, Logistics, Adaptive Testing,
Test Administration, and Test Scoring
Test Dissemination and Service Agreement
Lecture Lecture Role Play Lecture Role Play
TEST IMPLEMENTATION
13.
3.2 Administration
To quickly capture learning information, Tes Cepat is administered to a sample of 36 students in each school. In the period of 2016-2019, a total of 5,967 students in 410 KIAT Guru schools were randomly selected to take the test. These students were sampled based on the following rule. Six students from each grade, totaling to 36 students per school. This rule applies to both paper and digital formats of Tes Cepat. While the paper format requires a manual selection of students from each grade, the digital app can automatically generate lists of sample students for the test.
Tes Cepat should take place outside of school hours and can be administered in school or at home. To ensure students were comfortable in taking the test, implementation of Tes Cepat in the 410 schools did not interfere with school hours. Students were also allowed to take the test in school right after their last class, or at home following their afternoon rest. All testing locations and times must first be informed and approved by parents, and teachers if it was within school premises.
Tes Cepat adaptive flow is administered manually for the paper format, while automatically in the digital format. For the paper format, a handbook is provided for administrator as a reference to conduct the adaptive flow of the test. Figure 9 is a handbook for administering the reading adaptive test on the paper format. Meanwhile, the digital format automatically administered test items adaptively based on students’ responses. Both formats refer to the same adaptive logic as presented in section 2.1.3.
Due to its adaptive nature, Tes Cepat can be completed in five to ten minutes by each student. Students were only presented with test items within their competency range. The test stopped when they have answered the most difficult item within their capability. Some students however took longer time than the usual to work on each item. To prevent these students from burning out, time allowed to work on the test was limited
to fifteen minutes per subject. Test was stopped when student reached this time limit even if they had not answered their most difficult item. With this duration, it took an average of eighteen hours spread across three days to complete Tes Cepat in one school, conditional on the number of community volunteers conducting the test. The digital test has a feature that can automatically track and limit each student’s test taking duration.
For quality control, a designated volunteer should be assigned to monitor the testing procedure. Based on the number of available volunteers, and mobile phones for the digital format, Tes Cepat could be taken simultaneously for multiple students in one or two locations. For example, a village that had four community volunteers were able to test four students at a time, with each volunteer testing one student. Usually, they would conduct the test in the same location, with enough distance between each pair of volunteer and student. This arrangement was common and encouraged for time efficiency. However, quality control is imperative to make sure each volunteer was following the testing guideline, or to assist with any technical difficulties that may arise during the test. A community member or even teacher who have also attended the training and understands Tes Cepat, would then be assigned to monitor the test procedure, and to assist community volunteers or students with technical questions related to the test.
In the period of 2016-2019, a
total of 5,967 students
in 410 KIAT Guru schools
were randomly selected to
take the test.
14
.T
ES
T I
MP
LEM
EN
TA
TIO
N
Figu
re 9
. H
andb
ook
in A
dmin
iste
ring
Ada
ptiv
e Te
stin
g on
Rea
ding
Item
s in
Pap
er T
es C
epat
BAH
ASA
IND
ON
ESIA
STEP
1ST
EP 2
STEP
3 o
n ...
Stud
ent’s
gr
ade
?BA
HA
SA
test
be
gin
in
ques
tion
num
ber
....
CO
RREC
TIf
child
ans
wer
s C
ORR
ECTL
Y co
ntin
ue to
the
next
(hig
her n
umbe
r) ite
ms
until
chi
ld a
nsw
ers
inco
rrec
tly, m
ark
the
last
cor
rect
item
on
the
scor
ing
shee
t
INC
ORR
ECT
If ch
ild
answ
ers
inco
rrec
tly,
skip
to it
em
#:
If ch
ild
answ
ers
inco
rrec
tly,
skip
to it
em
#:
If ch
ild
answ
ers
inco
rrec
tly,
skip
to it
em
#:
If ch
ild
answ
ers
inco
rrec
tly,
skip
to it
em
#:
If ch
ild
answ
ers
inco
rrec
tly,
skip
to it
em
#:
If ch
ild
answ
ers
inco
rrec
tly,
skip
to it
em
#:
If ch
ild
answ
ers
inco
rrec
tly,
skip
to it
em
#:
If ch
ild
answ
ers
inco
rrec
tly,
skip
to it
em
#:
If ch
ild
answ
ers
inco
rrec
tly,
skip
to it
em
#:
If ch
ild s
till
answ
ers
inco
rrec
tly,
skip
to it
em
#:
17
(p
age
5)1
(pag
e 2)
STO
P
213
(p
age8
)9
(p
age
6)7
(p
age
5)1
(pag
e 2)
STO
P
319
(p
age
11)15
(p
age
9)13
(p
age
8)9
(p
age
6)7
(p
age
5)1
(pag
e 2)
STO
P
426
(p
age
15)
21
(pag
e 12
)19
(p
age
11)15
(p
age
9)13
(p
age8
)9
(p
age
6)7
(p
age
5)1
(pag
e 2)
STO
P
531
(p
age
19)
27
(pag
e 16
)26
(p
age
15)
21
(pag
e 12
)19
(p
age
11)15
(p
age
9)13
(p
age
8)9
(p
age
6)7
(p
age
5)1
(pag
e 2)
STO
P
TEST IMPLEMENTATION
15.
3.3 Actions from Results
Student abilities as indicated by Tes Cepat results are plotted manually for the paper format, and automatically for the digital format. For the paper format, administrators were provided with a simple
Figure 10. Digital Tes Cepat
Figure 11. Paper format Tes Cepat
template (Figure 10) that maps each student’s results to their grade-level equivalences. The digital format automatically generates maps of student’s results based on their grade-level competencies (Figure 11). Both formats refer to the same scoring method as presented in section 2.3.1.
Figure 12. Template to Map Student Ability from Paper Tes Cepat Results
Result Recapitulation Manual Form (If the computer is unavailable for recap)
Basic Compentencies Distribution based on Student’s amount
Grade AmountNumber
of Sample Student
Bahasa Indonesia Compentency
IL (0-1)
IL G (2-7)
G 1 (8-11)
G 2 (12-17)
G 3 (18-23)
G 4 (24-29)
G 5 (30-35)
G 6 (36-38)
1
2
3
4
5
IL = Iliterate G#: Grade-level competency
TEST IMPLEMENTATION
16.
Figure 13. Automated Map of Student Ability from Digital Tes Cepat
BAHASA INDONESIA
Grade IL IL G G 1 G 2 G 3 G 4 G 5 G 6 Sample
1 2 1 1 2 0 0 0 0 6
2 1 1 1 3 0 0 0 0 6
3 0 0 1 0 3 0 0 0 4
4 0 0 0 2 2 1 1 0 6
5 0 1 0 0 2 0 2 0 5
6 0 0 0 0 0 1 1 4 6
Note:
1. IL: Iliterate 3. BB: Below Basic
2. IN: Innumerate 4. G#: Grade-level competency
this information along with children’s aspirations were used by parents, community members, and teachers to develop a list of service indicators that each actor could do to assist their student’s learning in school and at home. Table 3 displays an example of service indicators that address community’s demand to add learning activities on reading and math in school and at home.
Tes Cepat results are used to develop service agreements between citizens and teachers to improve student learning outcomes. Learning information from Tes Cepat would be presented by community volunteers, usually in collaboration with principals, to a large forum attended by education stakeholders at the village level. In KIAT Guru schools,
Table 3. Service Agreement to Increase Reading and Math Learning in School and at Home based on Tes Cepat Results
Principal’s Service Indicators Class Teacher’s Service Indicators
Subject Teacher’s Service Indicators Parent’s Service Indicators
The Principal ensures that there is special guidance to improve students' reading, writing and arithmetic abilities as many as (...) times per month.
During (...) minutes, the class teacher trains reading/ writing and arithmetic/ addition/ multiplication/ division (...) times a week.
Subject teachers convey the topic of learning by asking children to read, write, and tell stories/ examples of problems or questions that stimulate students to count/ add/ multiply/ divide.
Parents provide children's learning tools at home such as poster letters, words and numbers or other learning media. Parents also familiarize children with Indonesian at home, especially when accompanying learning.
CONCLUSION
17.
Summary
Tes Cepat is built upon existing movements that give power to citizens, through access to information, to demand for quality education service delivery. The test is designed and developed as a citizen-led assessment on basic primary grade reading and math that results in actions by both citizens and teachers to improve learning outcomes in Indonesia’s remote schools. This background paper documents that:
As primary users of education services, parents and community members have limited means in measuring the quality of their children’s learning outcomes. Tes Cepat is a tool that is simple to administer and easy to understand, yet powerful for both citizens and teachers to act upon.
Dissemination of Tes Cepat results involves relevant stakeholder groups, including parents, community members, teachers, and village officials. They have the skills and motivations to implement the resulting actions. Longer term success of these actions depends on partnerships or collaborations across all relevant education stakeholders.
In KIAT Guru, resulting actions from Tes Cepat take form in the service indicators that community members and teachers agree upon. These indicators are consisted of actions that all relevant education stakeholders can do to improve student’s learning in school and at home.
Availability of both paper and digital Tes Cepat provides options for communities to choose which format is most suitable to the quality of their human resources and physical infrastructure, including internet connectivity.
Conclusion04
CONCLUSION
18.
4.2 Lessons from Paper dan Digital Tes Cepat Each format of Tes Cepat should be used according
to the quality of human resources and physical infrastructures available in targeted community. Both paper and digital formats of Tes Cepat have its own advantages and disadvantages. Despite operational differences, both formats comply to the test’s main principles. Table 4 lists several important conditions that differ in the implementation of paper and digital Tes Cepat. It comprises of varying advances, requirements, and limitations found in the training and administration of, as well as resulting actions from both formats of Tes Cepat. Future uses of the test should take this information into account in selection most suitable test format to pre-existing conditions of the targeted community.
Each format of Tes Cepat, paper and digital, has a set of unique preconditions, advantages as well as disadvantages in its implementation. Availability of human resources as well as physical infrastructure of targeted community must be taken into consideration when selecting which format of the test to use. Paper Tes Cepat is more suitable for areas with limited internet connectivity. Digital Tes Cepat is most efficient when used by community with basic digital literacy skills, as well as adequate availability of mobile devices that have access to internet.
Table 4. Differing Advantages and Disadvantages in Implementing Paper and Digital Tes Cepat
PAPER DIGITAL
Training • Requires more effort, time, and trainers to ensure quality control: sampling, test adaptivity, test instruction, test scoring.
• Not limited to availability of internet connection.
• Administrators are more interested in operating the application than understanding the objective of the assessment.
• Requires approximately 3 hours to conduct a special session to train how to operate android device itself, particularly for rural community setting.
• Limited availability of internet connection and mobile phone devices resulting in training can only be performed on a fixed number of participants.
Administration • Item bank is limited to what is provided in the test booklet.
• Human errors in adaptive testing and scoring.
• Item bank is expansive and can be updated through online synchronization feature.
• Sampling feature in the app allows community to have better ownership of the test sample selection.
• Requires less time as the app automated majority of the testing procedures.
• Automated instruction and adaptive testing reduce human errors.
• Limited to number of smart phones available in the village/ school. Majority of households in remote villages in Indonesia do not have smart phones.
• Technical errors in adaptive testing and scoring.
Dissemination of Test Results
• The process of interpreting test results requires more time and effort from community members as they need to manually input test results into a table that will help them map test results into a competency level. This results in lengthier dissemination preparation process.
• Manual test administration requires the community to have better comprehension of the types of questions at each grade level, which enables community to explain test processes, results and interpretation.
• Automated test scoring reduces human error and increases time efficiency of dissemination activity.
• The automation might limit community’s understanding of rationale behind interpretation of test results as produced by the application.
CONCLUSION
19.
4.3 Future Development
Based on the findings and lessons learned from Tes Cepat field implementation, the following are several considerations to be made for future and wider uses of the test:
Both government and non-government institutions that work in the education sector should consider providing similar tools that aid citizens to measure the quality of education service they are receiving. Information from this tool can be a powerful means to engage citizens in collaborating with other stakeholders, including teachers, to improve quality of education services in their local areas.
Although it was designed as an assessment tool for citizens, Tes Cepat which is similar to AKSI Sekolah (MoEC formative assessment initiative) may be utilized by teachers to conduct formative assessments and modify teaching pace accordingly.
Tes Cepat only measures learning outcomes of primary grade students in Indonesia’s remote areas. To optimize its scope of use, the test can be expanded to assess learning outcomes for secondary school students. Test items should measure more challenging cognitive and content domains to be more applicable for the national average learning outcomes. Tes Cepat framework can also be compared to that of other CLAs, which would allow for global comparability of measured learning outcomes through CLAs.
Digital Tes Cepat exposes community members in remote areas to technology. This is an important step to take considering the highly technology-based and connected society that Indonesia and the world are becoming. With the recent outbreak of COVID-19, the use of technology for remote learning is expected to gain more traction in the future.
Following the footsteps of other CLAs, future trainings on Tes Cepat should emphasize in detail actions required from the test results. Specifically, community members in KIAT Guru schools should be trained on developing appropriate service agreement indicators that correspond to the resulting learning information from the test. Alternately, community members and teachers can also be trained on steps to provide Teaching at the Right Level or equivalent to Pratham’s Reading Camps that are tailored student’s real competencies as measured by Tes Cepat.
21.
References
ASER. 2014. Annual status of education report (rural) 2013. New Delhi, India: ASER Centre.
Banerjee, A.V., Banerji, R., Duflo, E., Glennerster, R. and Khemani, S., 2010. Pitfalls of participatory programs: Evidence from a randomized evaluation in education in India. American Economic Journal: Economic Policy, 2(1), pp.1-30.
Banerjee, A. and Duflo, E. 2006. Addressing absence. Retrieved from http://economics.mit.edu/files/795
Barr, A., Mugisha, F., Serneels, P., & Zeitlin, A. 2012. Information and collective action in the community monitoring of schools: field and lab experimental evidence. Unpublished paper. Retrieved from https://www.tilburguniversity.edu/upload/397359d2-09a0-40d5-9502-8c3592aaae40_zeitlin.pdf
Bjorkman, M. & Svensson, J. 2009. When is community-based monitoring effective? evidence from a randomized experiment in primary health in Uganda. Retrieved from http://didattica.unibocconi.it/mypage/upload/49950_20091016_014406_JEEA_BJORKMANSVENSSON_REVISED.PDF
Carlitz, R. & Lipovsek, V. 2017. Citizen-led assessments and their effects on parents’ behavior. Retrieved from https://palnetwork.org/wp-content/uploads/2017/06/Citizen-led-assessments-and-their-effects-on-parents-behavior_Ruth-Carlitz-and-Dr.-Varja-Lipovsek.pdf
Dizon-Ross, Rebecca. 2016. Parents’ beliefs and children’s education: experimental evidence from malawi. Working Paper. http://faculty.chicagobooth.edu/rebecca.dizon-ross/research/index.html
Fazih, T., Afkar, R. & Tomlinson, H. 2018. Learning for all: towards quality education for enhanced productivity & economic growth in Indonesia. World Bank.
Gaduh, A., Pradhan, M., Priebe, J., and Susanti, D. 2020. Scores, Camera, Action? Incentivizing Teachers in Remote Areas. RISE Working Paper Series. 20/035. https://doi.org/10.35489/BSG-RISE-WP_2020/035.
Joshi, A. 2010. Do they work? Assessing the impact of transparency and accountability initiatives in service delivery. Sussex, UK: Institute of Development Studies.
OECD. (2019). PISA 2018 Assessment and Analytical Framework. PISA, OECD Publishing, Paris, https://doi.org/10.1787/b25efab8-en.
PAL Network. 2018. PAL Network 2018 Annual Plan: Assessment for Action. Retrieved from https://palnetwork.org/wp-content/uploads/2019/03/2018_PAL_Annual-Plan-Budget_Final.pdf
Pratham (n.d.). ASER Assessment and Survey Framework. Mumbai: Pratham Resource Centre.
REFERENCES
22.
Pusat Penilaian Pendidikan (Puspendik). (2012). 2012 Indonesian National Assessment Program. Kemampuan Membaca Siswa Kelas IV Sekolah Dasar: di Provinsi Kalimantan Timur dan D.I. Yogyakarta. Kementerian Pendidikan dan Kebudayaan, Badan Penelitian dan Pengembangan, Pusat Penilaian Pendidikan.
Pusat Penilaian Pendidikan (Puspendik). (2012). 2012 Indonesian National Assessment Program. Kemampuan Matematika Siswa Kelas IV Sekolah Dasar: di Provinsi Kalimantan Timur dan D.I. Read, L. & Atinc, T.M., 2017. Information for accountability: Transparency and citizen engagement for improved service delivery in education systems. Retrieved from https://palnetwork.org/wp-content/uploads/2017/06/Information_for_accountability_global_20170125.pdf
Ringold, D., Holla, A., Koziol, M. & Srinivasan, S. 2012. Citizens and service delivery: Assessing the use of social accountability approaches in human development. Washington, D.C.: The World Bank.
World Bank. 2019. Primary Education in Remote Indonesia: Survey Results from West Kalimantan and East Nusa Tenggara (English). Washington, DC: World Bank. http://documents.worldbank.org/curated/en/383471576785203185/Primary-Education-in-Remote-Indonesia-Survey-Results-from-West-Kalimantan-and-East-Nusa-Tenggara
23
.A
PP
EN
DIX
ES
Appe
ndix
1. R
eadi
ng a
nd M
ath
Com
pete
ncie
s m
easu
red
in T
es C
epat
Initi
al F
ram
ewor
k
Gra
deRe
adin
gM
ath
Dec
odin
gC
ompr
ehen
sion
Num
bers
Geo
met
ry &
Mea
sure
men
tSt
atis
tics
Gra
de 1
Lette
r, W
ord,
and
Sen
tenc
e Re
cogn
ition
Und
erst
andi
ng T
ext (
Elem
ents
, To
pic,
Mai
n Id
ea, K
ey
Info
rmat
ion)
Num
ber r
ecog
nitio
n, c
ount
ing,
se
quen
ces,
1–2-
digi
t add
ition
& s
ubtra
ctio
n
Gra
de 2
3-di
git a
dditi
on &
sub
tract
ion,
1–2-
digi
t mul
tiplic
atio
n &
divi
sion
Gra
de 3
3-di
git a
dditi
on &
sub
tract
ion
with
car
ryin
g/
borro
win
g,
1–2-
digi
t mul
tiplic
atio
n &
divi
sion
,
sim
ple
Frac
tion
Area
and
per
imet
er o
f re
ctan
gles
Gra
de 4
Addi
tion
& su
btra
ctio
n w
ith
Frac
tions
Gro
upin
g m
oney
,
perim
eter
of t
riang
les
Gra
de 5
Ratio
with
Fra
ctio
nsC
hara
cter
istic
s of
pris
m a
nd
cube
,
Area
of K
ite (q
uadr
ilate
ral),
Volu
me
of C
ube,
Mea
surin
g tim
e an
d sp
eed
Gra
de 6
Valu
e of
frac
tions
, Rat
io w
ith
Frac
tions
Area
of c
ircle
,
Volu
me
of c
ylin
der,
Sim
ple
Stat
istic
s,
Cal
cula
ting
mea
n
24
.
Appe
ndix
2. R
eadi
ng a
nd M
ath
Com
pete
ncie
s m
easu
red
in T
es C
epat
Exp
ande
d Fr
amew
ork
Gra
deRe
adin
gM
ath
Com
preh
ensi
onN
umbe
rsSe
quen
ces
Alg
ebra
Geo
met
ry &
M
easu
rem
ent
Stat
istic
s &
Prob
abili
tyRe
latio
ns &
Fu
nctio
ns
Gra
de 1
Wor
d Kn
owle
dge
Und
erst
andi
ng T
ext (
Elem
ents
, To
pic,
Mai
n Id
ea, K
ey
Info
rmat
ion)
Anal
yze
and
eval
uate
text
(G
enre
, Con
stru
ctio
n of
Se
nten
ces)
Eval
uate
text
(aut
hor c
onvi
ctio
n an
d pe
rspe
ctiv
e, e
vide
nce,
ar
gum
ents
)
Resp
ond
to te
xt (s
igni
fican
ce
to s
elf a
nd w
orld
, app
licat
ion,
in
tegr
ate
lear
ning
from
text
to
exis
ting
know
ledg
e)
Com
pare
and
sor
t, co
unt a
nd re
pres
ent
0-99
Sequ
ence
sAl
gebr
a ex
pres
sion
s,
sim
ilarit
ies,
num
ber
oper
atio
ns 1-
99
Shap
e re
cogn
ition
, le
ngth
and
tim
e m
easu
rem
ent
Gro
upin
g ob
ject
s ba
sed
on s
imila
r fe
atur
es
Gra
de 2
Com
pare
and
sor
t, co
unt a
nd re
pres
ent
100-
999,
sim
ple
fract
ions
Sequ
ence
sN
umbe
r ope
ratio
ns
100-
999,
sim
ple
mul
tiplic
atio
n
Shap
e re
cogn
ition
, le
ngth
and
tim
e m
easu
rem
ent
Sim
ple
conc
ept o
f pr
obab
ility
Gra
de 3
Prob
lem
s w
ith
num
ber o
pera
tions
10
0-99
9, 2
dig
its
mul
tiplic
atio
n an
d di
visi
on
Perim
eter
, gro
up
shap
es b
ased
on
sim
ilar f
eatu
res
Dis
play
dat
a on
a
tabl
e or
figu
re,
iden
tify
com
mon
re
al-li
fe p
roba
bilit
ies
Gra
de 4
Prim
e nu
mbe
rs, o
dd
and
even
inte
gers
, fra
ctio
ns to
dec
imal
s an
d pe
rcen
tage
s,
conc
epts
of r
atio
3 di
gits
mul
tiplic
atio
n an
d di
visi
on,
grea
test
com
mon
fa
ctor
(GC
F), l
east
co
mm
on m
ultip
le
(LC
M)
leng
th, w
eigh
t, an
d tim
e m
easu
rem
ent,
spee
d
Dis
play
dat
a on
a
tabl
e an
d gr
aph,
pr
obab
ility
of
occu
rrenc
es,
Gra
de 5
Roun
ding
dec
imal
s,
sort
fract
ions
, de
cim
als,
and
pe
rcen
tage
s, s
quar
e an
d cu
be ro
ots
GC
F an
d LC
M in
re
al li
fe p
robl
ems,
ad
ditio
n an
d su
btra
ctio
n us
ing
fract
ions
w
ith d
iffer
ent
deno
min
ator
s,
addi
tion
and
subt
ract
ion
with
de
cim
als
and
perc
enta
ges
Cub
e ne
ts, v
olum
e of
cub
esD
ispl
ay, c
ompa
re,
and
inte
rpre
t dat
aFu
nctio
ns a
nd
rela
tions
reco
gniti
on
and
repr
esen
tatio
n
Gra
de 6
Neg
ativ
e an
d po
sitiv
e in
tege
rsM
ultip
licat
ion
and
divi
sion
usi
ng
fract
ions
, dec
imal
s,
and
perc
enta
ges
Line
s in
circ
les,
pa
ram
eter
of s
hape
s w
ith n
on-s
traig
ht
lines
Cal
cula
ting
med
ian,
m
ode,
and
mea
n,
prob
abilit
y pr
oble
ms
with
frac
tions