THE WORLD BANK

Punjab Province, Pakistan

STUDENT ASSESSMENT SABER Province Report 2012

Key Policy Areas for Student Assessment Status

1. Classroom Assessment The 2006 National Curriculum provides very general, non-comprehensive guidelines on classroom assessment. Although available online, this document is not offered through other channels, such as teacher training courses or libraries, which teachers and others can easily access. There are some system-level mechanisms in place, including pre- and in-service teacher training opportunities, which are meant to ensure that teachers develop appropriate skills and expertise in classroom assessment. However, on-the-ground classroom assessment practices tend to vary from school to school, provide little feedback to students and parents, and are generally considered weak. There are no required uses of classroom assessment information to support student learning, and there are no mechanisms in place to systematically monitor the quality of classroom assessment practices.

2. Examinations The Secondary School Certificate exam is administered to students in Grades 9 and 10. The Punjab Boards of Intermediate and Secondary Education Act of 1976 authorizes the exam. The Exam Rules and Procedures document (known as the “Calendar”) for each Punjab Board covers key aspects of the exam. Officially, the examination is meant to measure student attainment of the knowledge and skills outlined in the 2006 National Curriculum. However, in practice, it assesses textbook knowledge. Moreover, the textbooks are based on an earlier (2002) curriculum, not the more recent (2006) one. There is a permanent staff dedicated to the exam, but it is insufficient to meet the needs of the examination due to staff members’ lack of technical capacity. Funding for the examination comes from student fees. Funding covers all core examination activities, but not research and development. There is also no technical documentation on the examination.

3. National Large-Scale Assessment (NLSA) The Punjab Education Assessment System (PEAS) was administered for the first time in 2011 as an independent, provincial-level assessment exercise. Previously, PEAS was part of the National Education Assessment System (NEAS). Under NEAS, Punjab administered the assessment to a provincially representative sample of students and published a provincial report separate from the national report. Although the PEAS office is a permanent agency, it is inadequately staffed to effectively carry out PEAS activities. This also was the case when PEAS was part of NEAS. PEAS results have been poorly disseminated and inadequately used.

4. International Large-Scale Assessment (ILSA) Punjab has not participated in an ILSA, and it does not have plans to do so in the near future.

2

Introduction The Punjab province in Pakistan has focused on increasing student learning outcomes by improving the quality of education in the province. An effective student assessment system is an important component to improving education quality and learning outcomes as it provides the necessary information to meet stakeholders’ decision-making needs. In order to gain a better understanding of the strengths and weaknesses of its existing student assessment system within the context of the province’s Public Sector Education system, the Punjab province decided to benchmark this system using standardized tools developed under The World Bank’s Systems Approach for Better Education Results (SABER) program. SABER is an evidence-based program to help countries systematically examine and strengthen the performance of different aspects of their education systems.

What is SABER-Student Assessment?

SABER-Student Assessment is a component of the SABER program that focuses specifically on benchmarking student assessment policies and systems. The goal of SABER-Student Assessment is to promote stronger assessment systems that contribute to improved education quality and learning for all. National governments and international agencies are increasingly recognizing the key role that assessment of student learning plays in an effective education system. The importance of assessment is linked to its role in: (i) providing information on levels of student

learning and achievement in the system; (ii) monitoring trends in education quality over

time; (iii) supporting educators and students with real-

time information to improve teaching and learning; and

(iv) holding stakeholders accountable for results.

SABER-Student Assessment methodology The SABER-Student Assessment framework is built on the available evidence base for what an effective assessment system looks like. The framework provides guidance on how countries can build more effective student assessment systems. The framework is structured around two main dimensions of assessment systems: the types/purposes of assessment activities and the quality of those activities. Assessment types and purposes Assessment systems tend to be comprised of three main types of assessment activities, each of which serves a different purpose and addresses different information needs. These three main types are: classroom assessment, examinations, and large-scale, system level assessments. Classroom assessment provides real-time information to support ongoing teaching and learning in individual classrooms. Classroom assessments use a variety of formats, including observation, questioning, and paper-and-pencil tests, to evaluate student learning, generally on a daily basis. Examinations provide a basis for selecting or certifying students as they move from one level of the education system to the next (or into the workforce). All eligible students are tested on an annual basis (or more often if the system allows for repeat testing). Examinations cover the main subject areas in the curriculum and usually involve essays and multiple-choice questions. Large-scale, system-level assessments provide feedback on the overall performance of the education system at particular grades or age levels. These assessments typically cover a few subjects on a regular basis (such as every 3 to 5 years), are often sample based, and use multiple-choice and short-answer formats. They may be national or international in scope. Appendix 1 summarizes the key features of these main types of assessment activities.

3

Quality drivers of an assessment system The key considerations when evaluating a student assessment system are the individual and combined quality of assessment activities in terms of the adequacy of the information generated to support decision making. There are three main drivers of information quality in an assessment system: enabling context, system alignment, and assessment quality. Enabling context refers to the broader context in which the assessment activity takes place and the extent to which that context is conducive to, or supportive of, the assessment. It covers such issues as the legislative or policy framework for assessment activities; institutional and organizational structures for designing, carrying out, or using results from the assessment; the availability of sufficient and stable sources of funding; and the presence of trained assessment staff. System alignment refers to the extent to which the assessment is aligned with the rest of the education system. This includes the degree of congruence between assessment activities and system learning goals, standards, curriculum, and pre- and in-service teacher training. Assessment quality refers to the psychometric quality of the instruments, processes, and procedures for the assessment activity. It covers such issues as design and implementation of assessment activities, analysis and interpretation of student responses to those activities, and the appropriateness of how assessment results are reported and used. Crossing the quality drivers with the different assessment types/purposes provides the framework and broad indicator areas shown in Table 1. This framework is a starting point for identifying indicators that can be used to review assessment systems and plan for their improvement.

Table 1: Framework for building an effective

assessment system, with indicator areas The indicators are identified based on a combination of criteria, including: professional standards for assessment; empirical research on the characteristics of effective

assessment systems, including analysis of the characteristics that differentiate between the assessment systems of low- versus high-performing nations; and

theory — that is, general consensus among experts that it contributes to effective assessment.

Levels of development The World Bank has developed a set of standardized questionnaires and rubrics for collecting and evaluating data on the three assessment types and related quality drivers. The questionnaires are used to collect data on the characteristics of the assessment system in a particular country or system. The information from the questionnaires is then applied to the rubrics in order to judge the development level of the country’s assessment system in different areas. The basic structure of the rubrics for evaluating data collected using the standardized questionnaires is summarized in Appendix 2. The goal of the rubrics is to provide a country with some sense of the

4

development level of its assessment activities compared to best or recommended practice in each area. For each indicator, the rubric displays four development levels—Latent, Emerging, Established, and Advanced. These levels are artificially constructed categories chosen to represent key stages on the underlying continuum for each indicator. Each level is accompanied by a description of what performance on the indicator looks like at that level. Latent is the lowest level of performance; it

represents absence of, or deviation from, the desired attribute.

Emerging is the next level; it represents partial presence of the attribute.

Established represents the acceptable minimum standard.

Advanced represents the ideal or current best practice.

A summary of the development levels for each assessment type is presented in Appendix 3. In reality, assessment systems are likely to be at different levels of development in different areas. For example, a system may be Established in the area of examinations, but Emerging in the area of large-scale, system-level assessment, and vice versa. While intuition suggests that it is probably better to be further along in as many areas as possible, the evidence is unclear as to whether it is necessary to be functioning at Advanced levels in all areas. Therefore, one might view the Established level as a desirable minimum outcome to achieve in all areas, but only aspire beyond that in those areas that most contribute to the national vision or priorities for education. In line with these considerations, the ratings generated by the rubrics are not meant to be additive across assessment types (that is, they are not meant to be added to create an overall rating for an assessment system; they are only meant to produce an overall rating for each assessment type). The methodology for assigning development levels is summarized in Appendix 4.

Education in the Punjab province Pakistan is a lower-middle-income country in South Asia. Current GDP per capita (US, 2012) is $1,257 with annual growth of 4 percent. Economic growth and progress in poverty reduction have slowed significantly over the past five years due to a weak macroeconomic environment, political and security conditions, and catastrophic natural shocks.

Punjab is Pakistan’s largest province (out of four), accounting for approximately three-fifths of the country’s population and income. In 2010/2011, primary net enrollment rate was 70 percent, and secondary net enrollment was 37 percent. In particular, there are low enrollment rates among girls in rural areas and children from poor households. In addition, the overall level of student achievement tends to be low; various assessments and surveys find that students perform significantly below curricular standards for key subjects and concepts at their grade level.

In order to increase both school participation and student achievement, Punjab has decided to focus specifically on improving teacher quality and performance. This decision is based on the government’s recognition that: 1) the teacher is the main instrument for student learning, and that many initiatives can affect the child only through the teacher; and 2) improved student learning helps retain students in school and attract new children to school. As a result, Punjab plans to promote teacher quality and performance both directly and indirectly through initiatives such as: (i) strengthening the system of field-based advisory support to teachers; (ii) introducing test-based recruitment of teachers; (iii) tying teacher compensation more closely to school performance; (iv) improving the regular collection of credible information on school, teacher, and student performance, and feeding this information back to internal and external stakeholders; and others.

5

Detailed information was collected on the Punjab province’s student assessment system using the SABER-Student Assessment questionnaires and rubrics in 2012. This SABER-Student Assessment report reflects the status of the student assessment system in the Punjab province as of 2012. It is important to remember that these tools primarily focus on benchmarking a system’s policies and arrangements for assessment activities at the system or macro level. Additional data would need to be collected to determine actual, on-the-ground practices in the Punjab province, particularly by teachers and students in schools. The following sections discuss the findings by each assessment type, accompanied by suggested policy options. The suggested policy options were determined in collaboration with key local stakeholders based on the Punjab province’s immediate interests and needs. Detailed, completed rubrics for each assessment type in the Punjab province are provided in Appendix 5.

Classroom Assessment Level of development: EMERGING In Punjab, the 2006 National Curriculum document provides general, non-comprehensive guidelines on classroom assessment. The document provides information on assessment purposes, types and questions, and includes some sample test content and scoring criteria. Although available online, this document is not offered through other channels that are more readily accessible by teachers and other stakeholders, such as through teacher training courses or libraries. The Punjab Curriculum Authority Act 2012 established the Curriculum Authority. The Curriculum Authority is responsible for overseeing the supervision of curricula, preparing and approving textbooks, and maintaining standards of education in the Punjab province. The 2006 National Curriculum document has not been provided to all practicing classroom teachers, and because not all teachers have access to the internet, many teachers are not aware of the guidelines provided in the document. In addition to the 2006 National Curriculum document, there are some system-wide resources available for teachers to engage in classroom

assessment activities, including textbooks which contain questions at the end of the chapters. The Taleemi Calendar is a document that contains student learning outcomes for each grade and subject. In addition, the Directorate of Staff Development (DSD) has developed Teacher’s Guides which contain Student Learning Outcome (SLO)- based lesson plans for each subject and grade in primary school. These guides also include a section on the kinds of activities and homework that teachers can use to assess students on the particular SLO. There are some system-level mechanisms, such as pre- and in-service teacher training opportunities, in place that are intended to ensure that teachers develop the appropriate skills and expertise in classroom assessment. School inspection and teacher supervision also include a component focused on classroom assessment. However, on-the-ground classroom assessment practices tend to be mainly about recalling information, provide little feedback to students and parents, and are generally considered weak. Teachers primarily develop the questions directly from the textbook and use the exercises at the end of the chapters; they do not use explicit criteria for scoring or grading students’ work. In general, classroom assessment activities tend to vary from school to school and are usually based on the textbooks and not on the curricula. However, textbooks tend to not be aligned with the curriculum because they are based on a previous version of the curriculum (developed in 2002) and not on the most recent curricula that went into effect in 2006. There are no required uses of classroom assessment information to support student learning, and there are no mechanisms in place to systematically monitor the quality of classroom assessment practices. Suggested policy options: 1. Make the curriculum document more easily

accessible to stakeholders through, for example, libraries, teacher training courses, and schools.

6

2. Develop detailed guidelines for teachers on how to carry out classroom assessment activities to support student learning.

3. Ensure more effective uses of classroom assessment information by: (i) communicating to teachers appropriate uses of the data; and (ii) making it compulsory for classroom assessment information to be disseminated to key stakeholders.

7

Examinations Level of development: EMERGING The Secondary School Certificate exam is administered to students in Grades 9 and 10. The Secondary School Certificate exam is a major examination for graduation from secondary school, with approximately 1.7 million students taking it every year, and is therefore the focus of this report. The Punjab Boards of Intermediate and Secondary Education Act of 1976 authorizes the exam and outlines the governance, distribution of power, and responsibilities among key stakeholders. Additionally, the Examination Rules and Procedures document (known as the “Calendar”) for each of the Punjab Boards covers key aspects of the exam including the rules for preparation, the format of the examination, and the procedures to investigate inappropriate behavior. The Secondary School Certificate examination is intended to measure student attainment of the knowledge and skills outlined in the 2006 National Curriculum. In practice, the examination tests textbook knowledge. At the same time, textbooks are not well aligned with the curricula due to two main issues: (1) some textbooks in circulation are based on a previous version of the national curricula developed in 2002 (new textbooks based on the new curricula developed in 2006 are in various stages of development), and (2) textbooks are not perceived to adequately cover all aspects of the curricula, such as higher order thinking skills. Thus, there is no clear link between the examination and the student learning outcomes or the skills identified in the curriculum, resulting in inadequate coverage and the lack of higher order skills tested. Efforts to improve the examination have been led by the government and by donors. For example, several efforts to improve aspects of the examinations have been made by the Boards along with the Education Department, including the introduction of multiple-choice and short answer questions. The Boards have also sought to address the issues of standardization between the exam boards and the variation in the scoring of papers by collectively setting papers,

computerizing the scoring of the multiple-choice questions, and swapping examination papers between boards. The Aga Khan University Examination Board and USAID have also supported efforts to improve the quality of the examination items and curriculum coverage of the Boards across Pakistan. Funding for the examination is provided by student fees, and covers all core examination activities including examination design, administration, data processing, and data reporting. There is a permanent staff responsible for running the examination, but it is insufficient to meet the needs of the examination due to staff members’ lack of technical capacity. Issues identified with the performance of the human resources responsible for the examination include errors in scoring, weakness in test design, omission of curricular topics, and errors in the examination questions and data processing. In general, comprehensive material to help prepare for the examination is accessible to most students. Such items include the textbook (which is considered the main resource, as the examination is textbook-based) and previous examination papers (as there is little repetition of examination questions from year to year). There is no technical documentation on the examination and there are no mechanisms in place to ensure the quality of the examination. Inappropriate behavior surrounding the examination process is moderate and includes leakage of the content of the paper prior to the examination, copying, and using unauthorized materials, such as prepared answers and notes. As a result, the credibility of the examination and its results tends to be called into question by many stakeholders. Suggested policy options: 1. Build the capacity of individuals responsible for

examination activities, especially in the areas of test design, curriculum alignment, and report writing. This could be done by: (i) making training opportunities available within the Boards, and (ii)

8

allocating funding for attending international courses or workshops. Introduce a mandate for capacity development of staff responsible for the examination. Identify which individuals or organizations will be responsible for the trainings, and provide them with regular funding for and access to the necessary training resources and key experts (at the provincial, national, and international levels).

2. Develop a technical report on the examination and disseminate it to key stakeholders.

3. Identify and agree on the official purposes and

possible uses of examination results, and introduce mechanisms, such as an oversight committee, to monitor actual uses.

9

National Large-Scale Assessment (NLSA) Level of development: EMERGING The Punjab province carries out several national large-scale assessments to students at the primary and secondary levels. The Punjab Examination Commission (PEC) was established in 2006 and awarded a statutory status in 2010 through an act of parliament. PEC is responsible for designing, implementing, and reporting on assessments for all Grade 5 (primary school) and Grade 8 (middle school) students in Punjab. PEC conducts Grade 5 and Grade 8 annual assessments (known as “examinations”) in all subjects for all the public and private schools in Punjab. PEC provides individual student reports which grade the student’s performance on each of the expected student learning outcomes (SLOs) by subject. The main objective of these examinations is to develop a key database about children’s performance and as a result measure the performance of the teachers and schools. The Punjab province also carries out the Punjab Education Assessment System (PEAS). PEAS was administered for the first time in 2011 as an independent, provincial-level assessment exercise administered to students in public sector schools. PEAS is viewed as the most important assessment for use by policymakers. Previously, PEAS was part of the National Education Assessment System (NEAS). Under NEAS, starting in 2005, Punjab administered the assessment to a provincially representative sample of students and published a provincial report separate from the national report. Regular funding for PEAS is allocated by the government and donors. Funding covers some core PEAS activities, but not research and development. Although the PEAS office is a permanent agency, it is inadequately staffed to effectively carry out all PEAS activities. In general, there are issues in test design, frequent errors in data processing, and delays due to the dependence on external human resources, including for paper development, data entry, and analysis. In the past, the University of Education (Lahore) offered short courses and a specialized course was offered by

the University of Melbourne to prepare PEAS staff. However, none of these courses has been sustained, and Punjab province currently offers no opportunities to prepare individuals for work on the PEAS. The PEAS assessment measures competencies identified in the curriculum. However, since assessments are based on the 2006 curricula and the textbooks available are based on the previous curricula (developed in 2002), some skills assessed on PEAS are not available in the textbooks and could cause some confusion amongst teachers who only use the textbook to teach students. Various mechanisms are in place to ensure the quality of PEAS including: (i) all administrators are trained according to a protocol, (ii) there is a standardized manual for the administrators, (iii) a pilot is conducted before the main data collection takes place, (iv) all booklets are numbered, and (v) external and internal reviewers and observers are employed. Coding and scoring is a centralized activity, conducted in one place and overseen by persons involved in data analysis. A two-day training is held for scorers, and scorers are also provided with detailed codebooks and rubrics for open-ended questions. 10 percent of the scored assessment papers are also scored by another person. Protocols are also in place for ensuring the quality of data entry and data cleaning. At the time of data collection for this report, the 2011 PEAS results had not been finalized and thus dissemination had not yet taken place. There are plans, however, to prepare and disseminate three types of reports (technical, thematic for different audiences, and a simplified report for a general audience) as well as to carry out district-based workshops to communicate the results. Suggested policy options: 1. Develop the capacity of PEAS staff by (i) introducing

workshops and trainings on education measurement and evaluation, including, in particular, data analysis and report writing; (ii) providing funding to PEAS staff to attend international programs on educational measurement and evaluation; and (iii) identifying

10

external human resources for conducting translation verifications between Urdu and Sindhi.

2. Identify the intended uses of PEAS results; develop and implement a plan to disseminate the results to stakeholders; introduce mechanisms, such as an oversight committee, to monitor actual uses of PEAS results.

.

11

International Large-Scale Assessment (ILSA) Level of development: LATENT The Punjab province has not participated in an ILSA, and it does not have plans to do so in the near future. Suggested policy options: 1. Create an opportunity for high-level discussion

among key stakeholders on key education policy questions or problems for which ILSA data could be useful.

2. Determine the need for, and possible next steps in relation to, participation in an ILSA exercise.

12

13

Appendix 1: Assessment Types and Their Key Differences

Classroom Large-scale assessment Surveys

Examinations

National International Exit Entrance Purpose

To provide immediate feedback to inform classroom instruction

To provide feedback on overall health of the system at particular grade/age level(s), and to monitor trends in learning

To provide feedback on the comparative performance of the education system at particular grade/age level(s)

To certify students as they move from one level of the education system to the next (or into the workforce)

To select students for further educational opportunities

Frequency Daily For individual subjects offered on a regular basis (such as every 3-5 years)

For individual subjects offered on a regular basis (such as every 3-5 years)

Annually and more often where the system allows for repeats

Annually and more often where the system allows for repeats

Who is tested?

All students Sample or census of students at a particular grade or age level(s)

A sample of students at a particular grade or age level(s)

All eligible students

All eligible students

Format Varies from observation to questioning to paper-and-pencil tests to student performances

Usually multiple choice and short answer

Usually multiple choice and short answer

Usually essay and multiple choice

Usually essay and multiple choice

Coverage of curriculum

All subject areas Generally confined to a few subjects

Generally confined to one or two subjects

Covers main subject areas

Covers main subject areas

Additional information collected from students?

Yes, as part of the teaching process

Frequently Yes Seldom Seldom

Scoring Usually informal and simple

Varies from simple to more statistically sophisticated techniques

Usually involves statistically sophisticated techniques

Varies from simple to more statistically sophisticated techniques

Varies from simple to more statistically sophisticated techniques

14

Appendix 2: Basic Structure of Rubrics for Evaluating Data Collected on a Student Assessment System

Dimension

Development Level

LATENT (Absence of, or deviation from,

attribute)

EMERGING (On way to meeting minimum standard)

ESTABLISHED (Acceptable

minimum standard)

ADVANCED (Best practice) Justification

EC—ENABLING CONTEXT EC1—Policies EC2—Leadership, public engagement

EC3—Funding EC4—Institutional arrangements EC5—Human resources

SA—SYSTEM ALIGNMENT SA1—Learning/quality goals SA2—Curriculum SA3—Pre-, in-service teacher training

AQ—ASSESSMENT QUALITY AQ1—Ensuring quality (design, administration, analysis)

AQ2—Ensuring effective uses

15

Appendix 3: Summary of the Development Levels for Each Assessment Type

Assessment Type LATENT EMERGING ESTABLISHED

ADVANCED

Absence of, or deviation from, the attribute

On way to meeting minimum standard

Acceptable minimum standard

Best practice

CLASSROOM ASSESSMENT

There is no system-wide institutional capacity to support and ensure the quality of classroom assessment practices.

There is weak system-wide institutional capacity to support and ensure the quality of classroom assessment practices.

There is sufficient system-wide institutional capacity to support and ensure the quality of classroom assessment practices.

There is strong system-wide institutional capacity to support and ensure the quality of classroom assessment practices.

EXAMINATIONS

There is no standardized examination in place for key decisions.

There is a partially stable standardized examination in place, and a need to develop institutional capacity to run the examination. The examination typically is of poor quality and is perceived as unfair or corrupt.

There is a stable standardized examination in place. There is institutional capacity and some limited mechanisms to monitor it. The examination is of acceptable quality and is perceived as fair for most students and free from corruption.

There is a stable standardized examination in place and institutional capacity and strong mechanisms to monitor it. The examination is of high quality and is perceived as fair and free from corruption.

NATIONAL (OR SYSTEM-LEVEL) LARGE-SCALE ASSESSMENT

There is no NLSA in place.

There is an unstable NLSA in place and a need to develop institutional capacity to run the NLSA. Assessment quality and impact are weak.

There is a stable NLSA in place. There is institutional capacity and some limited mechanisms to monitor it. The NLSA is of moderate quality and its information is disseminated, but not always used in effective ways.

There is a stable NLSA in place and institutional capacity and strong mechanisms to monitor it. The NLSA is of high quality and its information is effectively used to improve education.

INTERNATIONAL LARGE-SCALE ASSESSMENT

There is no history of participation in an ILSA nor plans to participate in one.

Participation in an ILSA has been initiated, but there still is need to develop institutional capacity to carry out the ILSA.

There is more or less stable participation in an ILSA. There is institutional capacity to carry out the ILSA. The information from the ILSA is disseminated, but not always used in effective ways.

There is stable participation in an ILSA and institutional capacity to run the ILSA. The information from the ILSA is effectively used to improve education.

16

Appendix 4: Methodology for Assigning Development Levels 1. The country team or consultant collects information about the assessment system in the country or system. 2. Based on the collected information, a level of development and score is assigned to each dimension in the rubrics:

Latent = 1 score point Emerging = 2 score points Established = 3 score points Advanced = 4 score points

3. The score for each quality driver is computed by aggregating the scores for each of its constituent dimensions. For example: The quality driver, ‘Enabling Context,’ in the case of ILSA, has 3 dimensions on which a hypothetical country or system receives the following scores: Dimension A = 2 points; Dimension B = 2 points; Dimension C = 3 points. The hypothetical country’s or system’s overall score for this quality driver would be: (2+2+3)/3 = 2.33 4. A preliminary level of development is assigned to each quality driver. 5. The preliminary development level is validated using expert judgment in cooperation with the country team and The World Bank Task Team Leader. For scores that allow a margin of discretion (i.e., to choose between two levels of development), a final decision has to be made based on expert judgment. For example, the aforementioned hypothetical country or system has an ‘Enabling Context’ score of 2.33, corresponding to a preliminary level of development of ‘Emerging or Established.’ Based on qualitative information not captured in the rubric, along with expert judgment, the country team chooses ‘Emerging’ as the most appropriate level.

6. Scores for certain key dimensions under ‘Enabling Context’ (in the case of EXAM, NLSA, and ILSA) and under ‘System Alignment’ (in the case of CLASS) were set as ceiling scores, i.e., the overall mean score for the particular assessment type cannot be greater than the score for these key dimensions. These key variables include formal policy, regular funding, having a permanent assessment unit, and the quality of assessment practices.

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

17

Appe

ndix

5: S

ABER

-Stu

dent

Ass

essm

ent R

ubri

cs P

unja

b Pr

ovin

ce, P

akis

tan

This

appe

ndix

pro

vide

s th

e co

mpl

eted

SAB

ER-S

tude

nt A

sses

smen

t rub

rics

for e

ach

type

of a

sses

smen

t act

ivity

in t

he P

unja

b pr

ovin

ce, P

akist

an. I

n ea

ch ro

w o

f th

e ru

bric

, the

rele

vant

sele

ctio

n is

indi

cate

d by

a th

ick

bord

er a

nd a

n as

teris

k. T

he s

elec

tion

may

incl

ude

a su

pers

crip

t num

ber t

hat r

efer

s to

the

just

ifica

tion

or

expl

anat

ion

for t

he se

lect

ion

(as i

ndic

ated

by

a th

ick

bord

er a

nd a

n as

teris

k). T

he e

xpla

natio

n or

just

ifica

tion

text

can

be

loca

ted

in th

e “D

evel

opm

ent-

leve

l rat

ing

just

ifica

tions

” se

ctio

n at

the

end

of e

ach

rubr

ic. I

f a r

ow in

clud

es a

sup

ersc

ript,

but n

ot a

thic

k bo

rder

and

an

aste

risk,

this

mea

ns th

at in

suffi

cien

t inf

orm

atio

n w

as a

vaila

ble

to d

eter

min

e th

e re

leva

nt se

lect

ion

in th

e ro

w.

PU

NJA

B PR

OVI

NCE

, PAK

ISTA

N Cl

assr

oom

Ass

essm

ent

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

18

ENAB

LIN

G CO

NTE

XT A

ND

SYS

TEM

ALI

GNM

ENT

Ove

rall

polic

y an

d re

sour

ce fr

amew

ork

with

in w

hich

cla

ssro

om a

sses

smen

t act

ivity

take

s pla

ce in

a c

ount

ry o

r sys

tem

, and

the

degr

ee to

whi

ch c

lass

room

as

sess

men

t act

ivity

is c

oher

ent w

ith o

ther

com

pone

nts o

f the

edu

catio

n sy

stem

. LA

TEN

T EM

ERGI

NG

ESTA

BLIS

HED

AD

VAN

CED

ENAB

LIN

G CO

NTE

XT A

ND

SYS

TEM

ALI

GNM

ENT

1:

Sett

ing

clea

r gu

idel

ines

for

clas

sroo

m a

sses

smen

t

Ther

e is

no s

yste

m-le

vel d

ocum

ent

that

pr

ovid

es

guid

elin

es

for

clas

sroo

m

asse

ssm

ent.

Ther

e is

an

info

rmal

sy

stem

-leve

l do

cum

ent

that

pro

vide

s gu

idel

ines

for

cl

assr

oom

ass

essm

ent. 1

Ther

e is

a fo

rmal

sys

tem

-leve

l doc

umen

t th

at p

rovi

des

guid

elin

es f

or c

lass

room

as

sess

men

t.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

The

avai

labi

lity

of

the

docu

men

t is

rest

ricte

d. 2

Th

e do

cum

ent i

s wid

ely

avai

labl

e.

ENAB

LIN

G CO

NTE

XT A

ND

SYS

TEM

ALI

GNM

ENT

2:

Alig

ning

clas

sroo

m a

sses

smen

t with

syst

em le

arni

ng g

oals

Ther

e ar

e no

sys

tem

-wid

e re

sour

ces

for

teac

hers

for c

lass

room

ass

essm

ent.

Ther

e ar

e sc

arce

sys

tem

-wid

e re

sour

ces

for t

each

ers f

or c

lass

room

ass

essm

ent.

Ther

e ar

e so

me

syst

em-w

ide

reso

urce

s fo

r tea

cher

s for

cla

ssro

om a

sses

smen

t. 3

Ther

e ar

e a

varie

ty

of

syst

em-w

ide

reso

urce

s av

aila

ble

for

teac

hers

fo

r cl

assr

oom

ass

essm

ent.

Ther

e is

no

offic

ial

curr

icul

um

or

stan

dard

s doc

umen

t. Th

ere

is an

of

ficia

l cu

rric

ulum

or

st

anda

rds

docu

men

t, bu

t it

is no

t cl

ear

wha

t st

uden

ts a

re e

xpec

ted

to le

arn

or

to w

hat l

evel

of p

erfo

rman

ce.

Ther

e is

an

offic

ial

curr

icul

um

or

stan

dard

s do

cum

ent

that

spe

cifie

s w

hat

stud

ents

are

exp

ecte

d to

lear

n, b

ut t

he

leve

l of

pe

rfor

man

ce

requ

ired

is no

t cl

ear. 4

Ther

e is

an

offic

ial

curr

icul

um

or

stan

dard

s do

cum

ent

that

spe

cifie

s w

hat

stud

ents

are

exp

ecte

d to

lea

rn a

nd t

o w

hat l

evel

of p

erfo

rman

ce.

ENAB

LIN

G CO

NTE

XT A

ND

SYS

TEM

ALI

GNM

ENT

3:

Hav

ing

effe

ctiv

e hu

man

res

ourc

es to

carr

y ou

t cla

ssro

om a

sses

smen

t act

iviti

es

Ther

e ar

e no

sys

tem

-leve

l m

echa

nism

s to

ens

ure

that

tea

cher

s de

velo

p sk

ills

and

expe

rtise

in c

lass

room

ass

essm

ent.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

ere

are

som

e sy

stem

-leve

l m

echa

nism

s to

en

sure

th

at

teac

hers

de

velo

p sk

ills

and

expe

rtise

in c

lass

room

as

sess

men

t. 5

Ther

e ar

e a

varie

ty

of

syst

em-le

vel

mec

hani

sms

to

ensu

re

that

te

ache

rs

deve

lop

skill

s an

d ex

pert

ise in

cla

ssro

om

asse

ssm

ent.

* * * *

*

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

19

ASSE

SSM

ENT

QU

ALIT

Y Q

ualit

y of

cla

ssro

om a

sses

smen

t des

ign,

adm

inist

ratio

n, a

naly

sis, a

nd u

se.

LATE

NT

EMER

GIN

G ES

TABL

ISH

ED

ADVA

NCE

D

ASSE

SSM

ENT

QU

ALIT

Y 1:

En

suri

ng th

e qu

alit

y of

clas

sroo

m a

sses

smen

t

Clas

sroo

m

asse

ssm

ent

prac

tices

su

ffer

from

wid

espr

ead

wea

knes

ses

or t

here

is

no i

nfor

mat

ion

avai

labl

e on

cla

ssro

om

asse

ssm

ent p

ract

ices

.

Clas

sroo

m

asse

ssm

ent

prac

tices

ar

e kn

own

to b

e w

eak.

6

Clas

sroo

m

asse

ssm

ent

prac

tices

ar

e kn

own

to b

e of

mod

erat

e qu

ality

. Cl

assr

oom

as

sess

men

t pr

actic

es

are

know

n to

be

gene

rally

of h

igh

qual

ity.

Ther

e ar

e no

mec

hani

sms

to m

onito

r the

qu

ality

of

cl

assr

oom

as

sess

men

t pr

actic

es. 7

Ther

e ar

e ad

hoc

mec

hani

sms

to m

onito

r th

e qu

ality

of

cl

assr

oom

as

sess

men

t pr

actic

es.

Ther

e ar

e lim

ited

syst

emat

ic m

echa

nism

s to

m

onito

r th

e qu

ality

of

cl

assr

oom

as

sess

men

t pra

ctic

es.

Ther

e ar

e va

ried

and

syst

emat

ic

mec

hani

sms

in

plac

e to

m

onito

r th

e qu

ality

of

cl

assr

oom

as

sess

men

t pr

actic

es.

ASSE

SSM

ENT

QU

ALIT

Y 2:

En

suri

ng e

ffect

ive

uses

of c

lass

room

ass

essm

ent

Clas

sroo

m a

sses

smen

t inf

orm

atio

n is

not

requ

ired

to

be

diss

emin

ated

to

ke

y st

akeh

olde

rs.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Cl

assr

oom

as

sess

men

t in

form

atio

n is

requ

ired

to b

e di

ssem

inat

ed to

som

e ke

y st

akeh

olde

rs. 8

Clas

sroo

m

asse

ssm

ent

info

rmat

ion

is re

quire

d to

be

diss

emin

ated

to

all

key

stak

ehol

ders

.

Ther

e ar

e no

req

uire

d us

es o

f cla

ssro

om

asse

ssm

ent t

o su

ppor

t stu

dent

lear

ning

. Th

ere

are

limite

d re

quire

d us

es

of

clas

sroo

m

asse

ssm

ent

to

supp

ort

stud

ent l

earn

ing.

Ther

e ar

e ad

equa

te

requ

ired

uses

of

cl

assr

oom

as

sess

men

t to

su

ppor

t st

uden

t le

arni

ng, e

xclu

ding

its

use

as a

n in

put f

or e

xter

nal e

xam

inat

ion

resu

lts.

Ther

e ar

e ad

equa

te

requ

ired

uses

of

cl

assr

oom

ass

essm

ent t

o su

ppor

t stu

dent

le

arni

ng, i

nclu

ding

its

use

as a

n in

put

for

exte

rnal

exa

min

atio

n re

sults

.

*

**

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

20

Clas

sroo

m A

sses

smen

t: Developmen

t-level rating j

ustifications

1.

The

only

sys

tem

-leve

l do

cum

ent

that

pro

vide

s so

me

reco

mm

enda

tions

on

asse

ssm

ent

prac

tices

is

the

Nat

iona

l Cu

rric

ula

2006

. It

prov

ides

gen

eral

in

form

atio

n on

ass

essm

ent p

urpo

ses,

type

s, a

nd q

uest

ions

and

som

e sa

mpl

e te

st c

onte

nt a

nd s

corin

g cr

iteria

, but

thes

e ar

e no

t co

mpr

ehen

sive

guid

elin

es.

Addi

tiona

lly, m

ost t

each

ers

have

not

bee

n pr

ovid

ed w

ith th

e cu

rric

ulum

doc

umen

t, an

d ar

e th

eref

ore

not a

war

e of

thes

e gu

idel

ines

. The

Pun

jab

Curr

icul

um

Auth

ority

Act

201

2 es

tabl

ished

the

Cur

ricul

um A

utho

rity.

The

Cur

ricul

um A

utho

rity

is re

spon

sible

for

over

seei

ng t

he s

uper

visio

n of

cur

ricul

um, p

repa

ring

and

appr

ovin

g te

xtbo

oks,

and

mai

ntai

ning

stan

dard

s of e

duca

tion

in th

e Pu

njab

pro

vinc

e.

2.

Th

e N

atio

nal C

urric

ula

2006

is a

vaila

ble

onlin

e bu

t not

thro

ugh

othe

r cha

nnel

s, su

ch a

s lib

rarie

s and

teac

her t

rain

ing

cour

ses a

nd c

olle

ges,

that

teac

hers

and

ot

her s

take

hold

ers c

an e

asily

acc

ess.

3.

The

Tale

emi C

alen

dar

is a

docu

men

t th

at c

onta

ins

stud

ent

lear

ning

out

com

es fo

r ea

ch g

rade

and

sub

ject

. The

re a

re a

lso t

extb

ooks

and

wor

kboo

ks t

hat

prov

ide

supp

ort

for

clas

sroo

m a

sses

smen

t. Th

ey c

onta

in q

uest

ions

at

the

end

of t

he c

hapt

er, a

s w

ell a

s gu

idel

ines

for

teac

hers

on

cond

uctin

g cl

assr

oom

ac

tiviti

es f

or a

sses

smen

t pu

rpos

es, s

uch

as s

ugge

stin

g to

tea

cher

s to

hol

d sp

eech

com

petit

ions

. Dire

ctor

ate

of S

taff

Deve

lopm

ent

(DSD

) ha

s de

velo

ped

Teac

her’s

Gui

des

that

con

tain

Stu

dent

Lea

rnin

g O

utco

me

(SLO

)-bas

ed le

sson

pla

ns fo

r eac

h su

bjec

t and

gra

de in

prim

ary

scho

ol. T

hese

incl

ude

a se

ctio

n on

th

e ki

nds o

f act

iviti

es/h

omew

ork

teac

hers

can

use

to a

sses

s stu

dent

s on

that

SLO

.

4.

Ther

e is

an o

ffici

al n

atio

nal c

urric

ulum

whi

ch w

ent

into

effe

ct in

200

6. A

fter

the

18th

con

stitu

tiona

l am

endm

ent,

the

auth

ority

to d

evel

op c

urric

ulum

has

be

en tr

ansf

erre

d to

the

prov

ince

s and

som

e pr

ovin

ces a

re d

evel

opin

g th

eir o

wn

curr

icul

a, b

ut it

is n

ot c

lear

whe

ther

the

Punj

ab p

rovi

nce

is do

ing

so.

5.

Sy

stem

-leve

l mec

hani

sms

incl

ude

pre-

and

in-s

ervi

ce te

ache

r tra

inin

g, a

s w

ell a

s sc

hool

insp

ectio

n or

teac

her s

uper

visio

n, w

hich

incl

udes

a c

ompo

nent

that

is

focu

sed

on c

lass

room

ass

essm

ent.

6.

It

is no

t co

mm

on f

or c

lass

room

ass

essm

ent

activ

ities

to

rely

mai

nly

on m

ultip

le-c

hoic

e, s

elec

tion-

type

que

stio

ns a

s sh

ort-

and

long

-ans

wer

que

stio

ns.

Clas

sroo

m a

sses

smen

t act

iviti

es a

re, h

owev

er, m

ainl

y ab

out r

ecal

ling

info

rmat

ion

beca

use

teac

hers

dev

elop

que

stio

ns d

irect

ly fr

om th

e te

xtbo

ok. T

each

ers

also

do

not

use

a pr

iori

crite

ria f

or g

radi

ng s

tude

nts'

wor

k be

caus

e th

ere

are

no s

yste

m-le

vel

guid

elin

es o

r sc

orin

g cr

iteria

. Ad

ditio

nally

, cl

assr

oom

as

sess

men

t ac

tiviti

es v

ary

from

sch

ool t

o sc

hool

and

gen

eral

ly p

rovi

de li

ttle

use

ful f

eedb

ack

to s

tude

nts

and

pare

nts.

In g

ener

al, c

lass

room

ass

essm

ent

activ

ities

are

bas

ed o

n th

e te

xtbo

oks n

ot th

e cu

rric

ula.

(Tex

tboo

ks a

re g

ener

ally

not

alig

ned

with

the

curr

icul

a be

caus

e th

e te

xtbo

oks i

n ci

rcul

atio

n ar

e ba

sed

on a

pre

viou

s ve

rsio

n of

the

natio

nal c

urric

ula

that

was

dev

elop

ed in

200

2. T

here

are

no

new

text

book

s th

at a

re b

ased

on

the

new

cur

ricul

a th

at w

ent i

nto

effe

ct in

200

6.)

7.

Th

ere

do n

ot a

ppea

r to

be a

ny s

yste

m-le

vel m

echa

nism

s th

at m

onito

r the

qua

lity

of a

sses

smen

t act

iviti

es. T

he fo

cus

is pr

imar

ily o

n te

st re

sults

. It a

ppea

rs

that

form

al te

ache

r pe

rfor

man

ce e

valu

atio

n is

only

a r

egul

ar p

ract

ice

at th

e se

cond

ary

leve

l and

in t

hat c

ase,

usu

ally

with

rel

atio

n to

Boa

rd e

xam

res

ults

. Ge

nera

lly, d

istric

t st

aff

visit

s te

nd t

o fo

cus

on m

ore

adm

inist

rativ

e as

pect

s, en

rollm

ent,

or t

each

er p

rese

nce.

At

the

key

grad

es a

nd t

he s

econ

dary

leve

l w

here

a b

oard

exa

m w

ill b

e ta

ken,

it a

ppea

rs th

ere

is an

inte

rest

in th

e st

uden

t res

ults

. Fin

ally

the

Dist

rict T

each

er E

duca

tors

(DTE

) visi

ts a

lso fo

cus o

n re

sults

of

thei

r mon

thly

test

s, no

t the

teac

hers

’ ow

n pr

actic

es.

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

21

8.

Te

ache

rs a

re r

equi

red

to r

epor

t on

indi

vidu

al s

tude

nts’

per

form

ance

to

scho

ol d

istric

t an

d M

inist

ry o

f Edu

catio

n of

ficia

ls. S

peci

fical

ly, s

tude

nt r

esul

ts a

re

repo

rted

to

the

dist

rict

educ

atio

n de

part

men

t (u

sual

ly f

or a

nnua

l tes

ts).

In s

ome

case

s fo

r th

ose

clas

ses

prep

arin

g fo

r bo

ard

exam

s, p

artic

ular

ly a

t th

e se

cond

ary

leve

l (i.e

., gr

ades

9 a

nd 1

0), i

t app

ears

that

term

test

s are

also

repo

rted

.

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

22

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

Exam

inat

ions

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

23

ENAB

LIN

G CO

NTE

XT

Ove

rall

fram

ewor

k of

pol

icie

s, le

ader

ship

, org

aniza

tiona

l str

uctu

res,

fisca

l and

hum

an re

sour

ces i

n w

hich

ass

essm

ent a

ctiv

ity ta

kes p

lace

in a

cou

ntry

or s

yste

m

and

the

exte

nt to

whi

ch th

at fr

amew

ork

is co

nduc

ive

to, o

r sup

port

ive

of, t

he a

sses

smen

t act

ivity

. LA

TEN

T EM

ERGI

NG

ESTA

BLIS

HED

AD

VAN

CED

ENAB

LIN

G CO

NTE

XT 1

:

Sett

ing

clea

r po

licie

s

No

stan

dard

ized

exam

inat

ion

has

take

n pl

ace.

Th

e st

anda

rdize

d ex

amin

atio

n ha

s be

en

oper

atin

g on

an

irreg

ular

bas

is.

The

exam

inat

ion

is a

stab

le p

rogr

am th

at

has b

een

oper

atin

g re

gula

rly.

This

optio

n do

es

not

appl

y to

th

is di

men

sion

Ther

e is

no

polic

y do

cum

ent

that

au

thor

izes t

he e

xam

inat

ion.

Th

ere

is an

in

form

al

or

draf

t po

licy

docu

men

t th

at

auth

orize

s th

e ex

amin

atio

n.

Ther

e is

a fo

rmal

pol

icy

docu

men

t th

at

auth

orize

s the

exa

min

atio

n. 1

Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

e po

licy

docu

men

t is

not

avai

labl

e to

th

e pu

blic

. 2

The

polic

y do

cum

ent

is av

aila

ble

to t

he

publ

ic.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

The

polic

y do

cum

ent

addr

esse

s so

me

key

aspe

cts o

f the

exa

min

atio

n. 3

The

polic

y do

cum

ent

addr

esse

s al

l ke

y as

pect

s of t

he e

xam

inat

ion.

ENAB

LIN

G CO

NTE

XT 2

: H

avin

g st

rong

lead

ersh

ip

All

stak

ehol

der

grou

ps s

tron

gly

oppo

se

the

exam

inat

ion

or a

re in

diffe

rent

to it

. 4

Mos

t st

akeh

olde

r gr

oups

op

pose

th

e ex

amin

atio

n.

Mos

t st

akeh

olde

rs g

roup

s su

ppor

t th

e ex

amin

atio

n.

All

stak

ehol

der

grou

ps

supp

ort

the

exam

inat

ion.

Ther

e ar

e no

att

empt

s to

im

prov

e th

e ex

amin

atio

n by

stak

ehol

der g

roup

s.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

ere

are

inde

pend

ent

atte

mpt

s to

im

prov

e th

e ex

amin

atio

n by

sta

keho

lder

gr

oups

. 5

Ther

e ar

e co

ordi

nate

d at

tem

pts

to

impr

ove

the

exam

inat

ion

by s

take

hold

er

grou

ps.

Effo

rts

to i

mpr

ove

the

exam

inat

ion

are

not

wel

com

ed

by

the

lead

ersh

ip

in

char

ge o

f the

exa

min

atio

n. 6

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Ef

fort

s to

im

prov

e th

e ex

amin

atio

n ar

e ge

nera

lly w

elco

med

by

the

lead

ersh

ip in

ch

arge

of t

he e

xam

inat

ion.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

(CO

NTI

NU

ED)

*

*

* **

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

24

LATE

NT

EMER

GIN

G ES

TABL

ISH

ED

ADVA

NCE

D

ENAB

LIN

G CO

NTE

XT 3

: H

avin

g re

gula

r fu

ndin

g

Ther

e is

no f

undi

ng a

lloca

ted

for

the

exam

inat

ion.

Th

ere

is irr

egul

ar f

undi

ng a

lloca

ted

for

the

exam

inat

ion.

7

Ther

e is

regu

lar f

undi

ng a

lloca

ted

for t

he

exam

inat

ion.

Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Fu

ndin

g co

vers

som

e co

re e

xam

inat

ion

activ

ities

: de

sign,

ad

min

istra

tion,

da

ta

proc

essin

g or

repo

rtin

g.

Fund

ing

cove

rs

all

core

ex

amin

atio

n ac

tiviti

es:

desig

n,

adm

inist

ratio

n,

data

pr

oces

sing

and

repo

rtin

g. 8

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Fu

ndin

g do

es n

ot c

over

res

earc

h an

d de

velo

pmen

t. 9

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Fu

ndin

g co

vers

re

sear

ch

and

deve

lopm

ent.

ENAB

LIN

G CO

NTE

XT 4

: H

avin

g st

rong

org

aniz

atio

nal s

truc

ture

s

The

exam

inat

ion

offic

e do

es n

ot e

xist

or

is ne

wly

est

ablis

hed.

Th

e ex

amin

atio

n of

fice

is ne

wly

es

tabl

ished

. Th

e ex

amin

atio

n of

fice

is a

stab

le

orga

niza

tion.

10

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

The

exam

inat

ion

offic

e is

not

acco

unta

ble

to

an

exte

rnal

bo

ard

or

agen

cy.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

e ex

amin

atio

n of

fice

is ac

coun

tabl

e to

an

ext

erna

l boa

rd o

r age

ncy.

11

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Exam

inat

ion

resu

lts a

re n

ot r

ecog

nize

d by

any

cer

tific

atio

n or

sele

ctio

n sy

stem

. Ex

amin

atio

n re

sults

are

rec

ogni

zed

by

cert

ifica

tion

or s

elec

tion

syst

em i

n th

e co

untr

y/sy

stem

. 12

Exam

inat

ion

resu

lts a

re r

ecog

nize

d by

on

e ce

rtifi

catio

n or

sel

ectio

n sy

stem

in

anot

her c

ount

ry/s

yste

m.

Exam

inat

ion

resu

lts a

re r

ecog

nize

d by

tw

o or

mor

e ce

rtifi

catio

n or

sel

ectio

n sy

stem

in a

noth

er c

ount

ry/s

yste

m.

The

exam

inat

ion

offic

e do

es n

ot h

ave

the

requ

ired

faci

litie

s to

car

ry o

ut t

he

exam

inat

ion.

13

The

exam

inat

ion

offic

e ha

s so

me

of t

he

requ

ired

faci

litie

s to

ca

rry

out

the

exam

inat

ion.

The

exam

inat

ion

offic

e ha

s al

l of

the

re

quire

d fa

cilit

ies

to

carr

y ou

t th

e ex

amin

atio

n.

The

exam

inat

ion

offic

e ha

s st

ate

of t

he

art f

acili

ties t

o ca

rry

out t

he e

xam

inat

ion.

(CO

NTI

NU

ED)

*

*

*

**

*

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

25

LATE

NT

EMER

GIN

G ES

TABL

ISH

ED

ADVA

NCE

D

ENAB

LIN

G CO

NTE

XT 5

:

Hav

ing

effe

ctiv

e hu

man

res

ourc

es

Ther

e is

no

staf

f to

ca

rry

out

the

exam

inat

ion.

Th

e ex

amin

atio

n of

fice

is in

adeq

uate

ly

staf

fed

to

effe

ctiv

ely

carr

y ou

t th

e ex

amin

atio

n, is

sues

are

per

vasiv

e. 1

4

The

exam

inat

ion

offic

e is

adeq

uate

ly

staf

fed

to

carr

y ou

t th

e ex

amin

atio

n ef

fect

ivel

y, w

ith m

inim

al is

sues

.

The

exam

inat

ion

offic

e is

adeq

uate

ly

staf

fed

to

carr

y ou

t th

e as

sess

men

t ef

fect

ivel

y, w

ith n

o iss

ues.

The

coun

try/

syst

em

does

no

t of

fer

oppo

rtun

ities

tha

t pr

epar

e fo

r w

ork

on

the

exam

inat

ion.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

e co

untr

y/sy

stem

of

fers

so

me

oppo

rtun

ities

tha

t pr

epar

e fo

r w

ork

on

the

exam

inat

ion.

The

coun

try/

syst

em o

ffers

a w

ide

rang

e of

opp

ortu

nitie

s th

at p

repa

re f

or w

ork

on th

e ex

amin

atio

n.

*

*

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

26

SYST

EM A

LIGN

MEN

T De

gree

to w

hich

the

asse

ssm

ent i

s coh

eren

t with

oth

er c

ompo

nent

s of t

he e

duca

tion

syst

em.

LATE

NT

EMER

GIN

G ES

TABL

ISH

ED

ADVA

NCE

D

SYST

EM A

LIGN

MEN

T 1:

Al

igni

ng e

xam

inat

ions

wit

h le

arni

ng g

oals

and

opp

ortu

nitie

s to

lear

n

It is

not

clea

r w

hat

the

exam

inat

ion

mea

sure

s.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

ere

is a

clea

r un

ders

tand

ing

of w

hat

the

exam

inat

ion

mea

sure

s.15

Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

Wha

t th

e ex

amin

atio

n m

easu

res

is qu

estio

ned

by s

ome

stak

ehol

der

grou

ps.

16

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

W

hat

is m

easu

red

by t

he e

xam

inat

ion

is la

rgel

y ac

cept

ed b

y st

akeh

olde

r gro

ups.

Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

Mat

eria

l to

prep

are

for

the

exam

inat

ion

is m

inim

al a

nd i

t is

only

acc

essib

le t

o ve

ry fe

w st

uden

ts.

Ther

e is

som

e m

ater

ial t

o pr

epar

e fo

r the

ex

amin

atio

n th

at i

s ac

cess

ible

to

som

e st

uden

ts.

Ther

e is

com

preh

ensiv

e m

ater

ial

to

prep

are

for

the

exam

inat

ion

that

is

acce

ssib

le to

mos

t stu

dent

s. 1

7

Ther

e is

com

preh

ensiv

e m

ater

ial

to

prep

are

for

the

exam

inat

ion

that

is

acce

ssib

le to

all

stud

ents

.

SYST

EM A

LIGN

MEN

T 2:

Pr

ovid

ing

teac

hers

wit

h op

port

uniti

es to

lear

n ab

out t

he e

xam

inat

ion

Ther

e ar

e no

cou

rses

or

wor

ksho

ps o

n ex

amin

atio

ns a

vaila

ble

to te

ache

rs.

Ther

e ar

e no

up

-to-

date

co

urse

s or

w

orks

hops

on

exam

inat

ions

ava

ilabl

e to

te

ache

rs.

Ther

e ar

e up

-to-

date

vol

unta

ry c

ours

es

or w

orks

hops

on

exam

inat

ions

ava

ilabl

e to

teac

hers

.

Ther

e ar

e up

-to-

date

com

pulso

ry c

ours

es

or

wor

ksho

ps

on

exam

inat

ions

fo

r te

ache

rs.

Teac

hers

ar

e ex

clud

ed

from

al

l ex

amin

atio

n-re

late

d ta

sks.

Te

ache

rs

are

invo

lved

in

ve

ry

few

ex

amin

atio

n-re

late

d ta

sks.

Te

ache

rs

are

invo

lved

in

so

me

exam

inat

ion-

rela

ted

task

s.

Teac

hers

ar

e in

volv

ed

in

mos

t ex

amin

atio

n-re

late

d ta

sks.

18

*

*

*

*

*

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

27

ASSE

SSM

ENT

QU

ALIT

Y De

gree

to w

hich

the

asse

ssm

ent m

eets

qua

lity

stan

dard

s, is

fair,

and

is u

sed

in a

n ef

fect

ive

way

. LA

TEN

T EM

ERGI

NG

ESTA

BLIS

HED

AD

VAN

CED

ASSE

SSM

ENT

QU

ALIT

Y 1:

En

suri

ng q

ualit

y

Ther

e is

no t

echn

ical

rep

ort

or o

ther

do

cum

enta

tion.

Th

ere

is so

me

docu

men

tatio

n on

the

ex

amin

atio

n, b

ut i

t is

not

in

a fo

rmal

re

port

form

at.

Ther

e is

a co

mpr

ehen

sive

tech

nica

l re

port

but

with

rest

ricte

d ci

rcul

atio

n.

Ther

e is

a co

mpr

ehen

sive,

hig

h qu

ality

te

chni

cal r

epor

t av

aila

ble

to t

he g

ener

al

publ

ic.

Ther

e ar

e no

mec

hani

sms

in p

lace

to

ensu

re th

e qu

ality

of t

he e

xam

inat

ion.

Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

Ther

e ar

e lim

ited

syst

emat

ic m

echa

nism

s in

pla

ce t

o en

sure

the

qua

lity

of t

he

exam

inat

ion.

Ther

e ar

e va

ried

and

syst

emat

ic

mec

hani

sms

in

plac

e to

en

sure

th

e qu

ality

of t

he e

xam

inat

ion.

ASSE

SSM

ENT

QU

ALIT

Y 2:

En

suri

ng fa

irne

ss

Inap

prop

riate

beh

avio

r su

rrou

ndin

g th

e ex

amin

atio

n pr

oces

s is h

igh.

In

appr

opria

te b

ehav

ior

surr

ound

ing

the

exam

inat

ion

proc

ess i

s mod

erat

e. 1

9 In

appr

opria

te b

ehav

ior

surr

ound

ing

the

exam

inat

ion

proc

ess i

s low

. In

appr

opria

te b

ehav

ior

surr

ound

ing

the

exam

inat

ion

proc

ess i

s mar

gina

l.

The

exam

inat

ion

resu

lts l

ack

cred

ibili

ty

for a

ll st

akeh

olde

r gro

ups.

Th

e ex

amin

atio

n re

sults

are

cre

dibl

e fo

r so

me

stak

ehol

der g

roup

s. 2

0 Th

e ex

amin

atio

n re

sults

are

cre

dibl

e fo

r al

l sta

keho

lder

gro

ups.

Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

The

maj

ority

of

the

stud

ents

(ov

er 5

0%)

may

not

take

the

exam

inat

ion

beca

use

of

lang

uage

, ge

nder

, or

oth

er e

quiv

alen

t ba

rrie

rs.

A sig

nific

ant

prop

ortio

n of

st

uden

ts

(10%

-50%

) may

not

take

the

exam

inat

ion

beca

use

of l

angu

age,

gen

der,

or o

ther

eq

uiva

lent

bar

riers

.

A sm

all p

ropo

rtio

n of

stu

dent

s (le

ss th

an

10%

) m

ay

not

take

th

e ex

amin

atio

n be

caus

e of

lan

guag

e, g

ende

r, or

oth

er

equi

vale

nt b

arrie

rs. 2

1

All

stud

ents

can

tak

e th

e ex

amin

atio

n;

ther

e ar

e no

lang

uage

, ge

nder

or

othe

r eq

uiva

lent

bar

riers

.

(CO

NTI

NU

ED)

* *

* *

*

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

28

LA

TEN

T EM

ERGI

NG

ESTA

BLIS

HED

AD

VAN

CED

ASSE

SSM

ENT

QU

ALIT

Y 3:

Us

ing

exam

inat

ion

info

rmat

ion

in a

fair

way

Exam

inat

ion

resu

lts a

re n

ot u

sed

in a

pr

oper

way

by

all s

take

hold

er g

roup

s.

Exam

inat

ion

resu

lts a

re u

sed

by s

ome

stak

ehol

der g

roup

s in

a pr

oper

way

. Ex

amin

atio

n re

sults

are

use

d by

mos

t st

akeh

olde

r gro

ups i

n a

prop

er w

ay. 2

2 Ex

amin

atio

n re

sults

ar

e us

ed

by

all

stak

ehol

der g

roup

s in

a pr

oper

way

.

Stud

ent n

ames

and

resu

lts a

re p

ublic

. Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

Stud

ents

’ res

ults

are

con

fiden

tial.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

ASSE

SSM

ENT

QU

ALIT

Y 4:

En

suri

ng p

ositi

ve c

onse

quen

ces o

f the

exa

min

atio

n

Ther

e ar

e no

opt

ions

for

stu

dent

s w

ho

do n

ot p

erfo

rm w

ell o

n th

e ex

amin

atio

n,

or s

tude

nts

mus

t le

ave

the

educ

atio

n sy

stem

.

Ther

e ar

e ve

ry

limite

d op

tions

fo

r st

uden

ts w

ho d

o no

t per

form

wel

l on

the

exam

inat

ion.

Ther

e ar

e so

me

optio

ns fo

r stu

dent

s who

do

not

per

form

wel

l on

the

exam

inat

ion.

23

Ther

e is

a va

riety

of o

ptio

ns fo

r stu

dent

s w

ho

do

not

perf

orm

w

ell

on

the

exam

inat

ion.

Ther

e ar

e no

mec

hani

sms

in p

lace

to

mon

itor

the

cons

eque

nces

of

th

e ex

amin

atio

n.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

ere

are

som

e m

echa

nism

s in

pla

ce t

o m

onito

r th

e co

nseq

uenc

es

of

the

exam

inat

ion.

Ther

e is

a va

riety

of m

echa

nism

s in

pla

ce

to

mon

itor

the

cons

eque

nces

of

th

e ex

amin

atio

n.

*

*

*

*

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

29

Exam

inat

ions

: Developmen

t-level rating j

ustifications

1.

The

form

al p

olic

y do

cum

ent

is th

e Pu

njab

Boa

rds

of In

term

edia

te a

nd S

econ

dary

Edu

catio

n Ac

t of

197

6. A

dditi

onal

ly, t

here

are

"Ex

amin

atio

n Ru

les

and

Proc

edur

es" d

ocum

ents

that

are

kno

wn

as th

e "C

alen

dar"

for e

ach

of th

e Pu

njab

Boa

rds,

whi

ch c

over

s key

are

as.

2.

Al

l act

s, o

rdin

ance

s, a

nd th

e Ca

lend

ar a

re te

chni

cally

pub

lic d

ocum

ents

, but

they

are

not

nec

essa

rily

easil

y ac

cess

ible

. The

full

Cale

ndar

is a

vaila

ble

in h

ard

copy

at t

he B

oard

on

requ

est,

and

it m

ust b

e pu

rcha

sed.

Par

ts o

f the

Cal

enda

r are

eas

ily a

cces

sible

on

the

web

site,

but

inte

rnet

is n

ot e

asily

acc

essib

le to

all

stak

ehol

ders

.

3.

The

Punj

ab B

oard

s of I

nter

med

iate

and

Sec

onda

ry E

duca

tion

Act o

utlin

e th

e go

vern

ance

, dist

ribut

ion

of p

ower

, and

resp

onsib

ilitie

s am

ong

key

stak

ehol

ders

. Th

e Ca

lend

ar o

utlin

es th

e pr

oced

ures

to in

vest

igat

e an

d ad

dres

s se

curit

y br

each

es, c

heat

ing,

or o

ther

form

s of

inap

prop

riate

beh

avio

r, th

e pr

oced

ures

for

spec

ial/d

isadv

anta

ged

stud

ents

, an

d sp

ecifi

es w

ho c

an s

it fo

r th

e ex

amin

atio

n. I

t al

so i

dent

ifies

the

rul

es a

bout

the

pre

para

tion

and

form

at o

f th

e ex

amin

atio

n.

4.

Th

ere

is lit

tle p

ublic

ly a

vaila

ble

evid

ence

on

stak

ehol

der s

uppo

rt o

r opp

ositi

on, a

nd th

ere

is ve

ry li

ttle

pub

lic d

ebat

e on

this

issue

.

5.

Effo

rts

to im

prov

e th

e ex

amin

atio

n ha

ve b

een

led

by t

he g

over

nmen

t an

d, m

ost

rece

ntly

, by

dono

rs. F

or e

xam

ple,

the

re h

ave

been

sev

eral

effo

rts

to

impr

ove

diffe

rent

asp

ects

of

the

exam

inat

ions

by

the

Boar

ds a

long

with

the

Edu

catio

n De

part

men

t. Th

e In

ter

Boar

d Co

mm

ittee

of

Chai

rmen

(IB

CC)

has

chan

ged

the

pape

r pa

tter

n in

rec

ent y

ears

to in

clud

e m

ultip

le-c

hoic

e qu

estio

ns a

nd s

hort

ans

wer

que

stio

ns. U

nder

the

guid

ance

of t

he C

hief

Min

ister

last

ye

ar,

they

cre

ated

a t

alen

t po

ol o

f be

st p

aper

set

ters

and

pro

vide

d ca

paci

ty d

evel

opm

ent

to t

hem

. Th

e Bo

ards

hav

e so

ught

to

addr

ess

issue

s of

st

anda

rdiza

tion

betw

een

the

exam

boa

rds

by s

ettin

g pa

pers

col

lect

ivel

y. T

hey

have

also

sou

ght t

o de

al w

ith v

aria

tions

in th

e sc

orin

g of

the

pape

rs th

roug

h pr

oces

ses

such

as

com

pute

rizin

g th

e sc

orin

g of

the

mul

tiple

-cho

ice

ques

tion

sect

ions

, sw

appi

ng p

aper

s be

twee

n Bo

ards

and

syn

dica

ted

mar

king

of o

pen-

ende

d qu

estio

ns (i

n w

hich

one

per

son

mar

ks o

ne q

uest

ion

and

the

othe

r the

nex

t to

min

imize

effe

cts

of th

e m

arke

r on

the

over

all s

core

). Th

e Bo

ards

hav

e al

so so

ught

to c

ompu

teriz

e th

e sy

stem

from

regi

stra

tion

to sc

orin

g to

resu

lt pr

oduc

tion;

alth

ough

ther

e ar

e iss

ues w

ith th

is sy

stem

cur

rent

ly, t

his m

ay in

the

futu

re re

duce

err

ors i

n th

e re

sult

and

inst

ance

s of u

nfai

r mea

ns. T

he P

unja

b Ed

ucat

ion

Depa

rtm

ent a

nd th

e Bo

ards

hav

e in

stitu

ted

seve

ral m

easu

res t

o cu

rb

chea

ting

and

othe

r in

appr

opria

te b

ehav

iors

. Fi

nally

, th

ere

appe

ar t

o be

effo

rts

by t

he A

ga K

han

Uni

vers

ity E

xam

inat

ion

Boar

d (A

KUEB

) an

d a

USA

ID-

supp

orte

d ED

LIN

KS p

roje

ct to

impr

ove

the

qual

ity o

f the

exa

m p

aper

item

s an

d cu

rric

ulum

cov

erag

e of

the

Boar

ds a

cros

s Pa

kist

an. H

owev

er th

is is

a fa

irly

new

initi

ativ

e an

d ap

art f

rom

a w

orks

hop

with

pap

er s

ette

rs—

indi

vidu

als

who

set

the

exam

inat

ion

ques

tion

pape

rs—

from

the

Boar

ds, i

t is

not c

lear

wha

t ot

her a

ctiv

ities

hav

e be

en im

plem

ente

d.

6.

Th

e Bo

ards

at t

he h

ighe

r lev

els

appe

ar to

be

awar

e of

the

vario

us is

sues

in th

e ex

amin

atio

n sy

stem

and

as

a re

sult

have

take

n st

eps

to a

ddre

ss th

e iss

ues

them

selv

es. H

owev

er, s

ince

effo

rts

to im

prov

e th

e ex

amin

atio

n fr

om o

utsid

e th

e go

vern

men

t, su

ch a

s th

ose

by th

e AK

U-E

B, h

ave

not o

ccur

red,

it is

diff

icul

t to

say

whe

ther

they

wou

ld b

e re

cept

ive

to su

ch c

hang

es.

7.

Th

e fu

ndin

g fo

r th

e ex

amin

atio

n ar

e fr

om s

tude

nt f

ees

colle

cted

in p

revi

ous

year

s an

d ot

her

fees

for

issu

ing

or v

alid

atin

g ce

rtifi

cate

s, c

ondu

ctin

g a

re-

eval

uatio

n of

pap

ers,

etc.

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

30

8.

Da

ta a

naly

sis a

nd re

port

ing

are

limite

d to

dev

elop

men

t and

repo

rtin

g of

exa

m re

sults

for i

ndiv

idua

l can

dida

tes.

9.

Onl

y th

e La

hore

Boa

rd h

as a

rese

arch

sec

tion;

how

ever

, thi

s se

ctio

n do

es n

ot n

eces

saril

y co

nduc

t res

earc

h. It

s fu

nctio

ns a

re u

sual

ly to

dev

elop

des

crip

tive

anal

ysis

of e

xam

res

ults

(i.e

., by

gen

der

or lo

catio

n) a

nd h

andl

e no

n-re

sear

ch a

ctiv

ities

suc

h as

sch

olar

ship

s, d

ebat

es, a

nd e

ssay

com

petit

ions

. The

re is

no

staf

f dev

elop

men

t.

10

. The

old

est

Boar

d, t

he L

ahor

e Bo

ard

of S

econ

dary

& In

term

edia

te E

duca

tion,

was

est

ablis

hed

in 1

954,

whi

le t

he m

ost

rece

nt B

oard

, the

Der

a Gh

azi K

han

Boar

d of

Sec

onda

ry &

Inte

rmed

iate

Edu

catio

n, w

as e

stab

lishe

d in

198

9.

11

. Acc

ount

able

to th

e De

part

men

t of H

ighe

r Edu

catio

n, P

unja

b.

12

. The

IBCC

/For

eign

Min

istry

att

ests

to th

e us

e of

the

SSC

exam

cer

tific

ates

abr

oad.

Alth

ough

the

SSC

exam

wou

ld n

ot b

e us

ed fo

r adm

issio

n in

to u

nive

rsity

per

se

, it w

ould

pre

sum

ably

form

par

t of t

he se

cond

ary

scho

ol e

duca

tion

reco

rd fo

r adm

issio

ns.

13

. The

con

sulta

nt o

nly

visit

ed o

ne L

ahor

e BI

SE p

hysic

ally

, an

d th

e se

lect

ions

for

thi

s qu

estio

n ar

e ba

sed

on t

his

Boar

d, e

ither

thr

ough

obs

erva

tion

or

resp

onde

nt c

omm

ents

. Giv

en th

at th

e Bo

ards

are

loca

ted

in v

ery

diffe

rent

are

as, i

t is

diffi

cult

to d

eter

min

e w

heth

er th

ese

resp

onse

s in

the

ques

tionn

aire

, pa

rtic

ular

ly fo

r tec

hnol

ogic

al fa

cilit

ies,

are

repr

esen

tativ

e of

the

othe

r Boa

rds a

s wel

l.

14. W

hile

the

re is

per

man

ent

or f

ull-t

ime

staf

f, it

is in

suffi

cien

t to

mee

t th

e ne

eds

of t

he e

xam

inat

ion.

For

exa

mpl

e, t

here

is la

ck o

f st

aff

who

can

per

form

re

sear

ch a

nd d

ata

anal

ysis,

and

lack

of

tech

nica

l sta

ff w

ith s

kills

in e

xam

inat

ion

desig

n to

gui

de t

he p

aper

dev

elop

men

t pr

oces

s. Is

sues

tha

t ha

ve b

een

iden

tifie

d w

ith t

he p

erfo

rman

ce o

f the

hum

an r

esou

rces

res

pons

ible

for

the

exam

inat

ion

incl

ude

erro

rs in

sco

ring

that

hav

e le

d to

del

ays

in r

esul

ts b

eing

re

port

ed, w

eakn

ess i

n te

st d

esig

n, o

miss

ion

of c

urric

ular

topi

cs, a

nd fr

eque

nt e

rror

s in

the

exam

inat

ion

ques

tions

and

dat

a pr

oces

sing.

15. O

ffici

ally

, the

exa

min

atio

n se

eks t

o m

easu

re th

e na

tiona

l cur

ricul

um. H

owev

er, t

here

is n

o cl

ear l

ink

to th

e st

uden

t lea

rnin

g ou

tcom

es o

r the

ski

lls id

entif

ied

in th

e cu

rric

ulum

, ofte

n re

sulti

ng in

inad

equa

te c

over

age

and

the

lack

of h

ighe

r ord

er s

kills

test

ed a

s no

ted

by re

sear

ch s

tudi

es. I

n pr

actic

e, it

is e

ssen

tially

th

e te

xtbo

ok c

onte

nt th

at is

mea

sure

d by

the

exam

(in

Punj

ab, t

here

is o

ne s

et o

f pro

vinc

ially

app

rove

d te

xtbo

oks

in s

choo

ls). T

extb

ooks

can

not b

e sa

id to

be

wel

l alig

ned

with

the

cur

ricul

a: (

1) S

ome

text

book

s in

circ

ulat

ion

are

base

d on

a p

revi

ous

vers

ion

of t

he n

atio

nal c

urric

ula

deve

lope

d in

200

2 (n

ew

text

book

s ba

sed

on th

e ne

w c

urric

ula,

i.e.

200

6, a

re in

var

ious

sta

ges

of d

evel

opm

ent)

, and

(2) t

hey

are

not p

erce

ived

to a

dequ

atel

y co

ver a

ll as

pect

s of

the

curr

icul

a, su

ch a

s hig

her o

rder

thin

king

skill

s.

16. D

iffer

ent

stak

ehol

ders

hav

e di

ffere

nt c

once

rns.

Man

y st

akeh

olde

rs (s

uch

as p

aren

ts, s

tude

nts,

gen

eral

pub

lic) a

re m

ore

conc

erne

d w

ith is

sues

rel

ated

to

chea

ting

and

spee

d, a

nd a

ccur

acy

of s

corin

g. H

owev

er, t

hose

who

reco

gnize

that

ther

e ar

e in

adeq

uaci

es in

the

curr

icul

um c

over

age

and

skill

s te

sted

do

not

acce

pt w

hat i

s m

easu

red.

Suc

h re

spon

dent

s fo

r thi

s st

udy

have

incl

uded

the

Chai

rman

of a

Boa

rd, s

taff

at N

GOs

and

prof

essio

nal d

evel

opm

ent i

nstit

utio

ns,

and

som

e ed

ucat

ors.

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

31

17. M

ater

ials

requ

ired

for p

repa

ring

for t

he e

xam

incl

ude

the

text

book

(the

mai

n re

sour

ce, a

s th

e ex

am is

ess

entia

lly te

xtbo

ok-b

ased

) and

pas

t pap

ers

(as

ther

e is

appa

rent

ly n

ot a

lot o

f rep

etiti

on in

exa

m q

uest

ions

from

yea

r to

year

). Te

xtbo

oks

in g

over

nmen

t sch

ools

are

now

pro

vide

d fr

ee o

f cos

t. Re

spon

dent

s fe

el

that

bot

h th

ese

reso

urce

s sh

ould

be

read

ily a

vaila

ble

in th

e m

arke

t eve

n in

rura

l are

as, a

nd c

ost i

s no

t too

hig

h ev

en fo

r tho

se w

ho h

ave

to p

ay. T

hus o

nly

a ve

ry s

mal

l per

cent

age

may

hav

e di

fficu

lty a

cces

sing

thes

e ite

ms.

The

mat

eria

l on

the

exam

inat

ion

incl

udes

exa

mpl

es o

f the

type

s of

que

stio

ns th

at a

re o

n th

e ex

amin

atio

n, p

ast e

xam

inat

ion

pape

rs, a

nd e

xam

ples

of e

xam

inat

ion

pape

rs d

evel

oped

by

the

Boar

ds.

18

. Ret

ired

teac

hers

sele

ct o

r cre

ate

exam

inat

ion

ques

tions

, whi

le a

ctiv

e te

ache

rs a

dmin

ister

and

scor

e th

e ex

amin

atio

n, su

perv

ise

the

exam

inat

ion

proc

edur

es,

and

act a

s a ju

dge

in, f

or e

xam

ple,

ora

ls an

d pr

actic

al.

19

. Var

ious

inap

prop

riate

beh

avio

rs d

imin

ish th

e cr

edib

ility

of t

he e

xam

inat

ions

. Suc

h be

havi

ors

incl

ude

leak

age

of th

e ex

amin

atio

n pa

per

cont

ent o

r par

t of a

pa

per p

rior t

o th

e ex

amin

atio

n, im

pers

onat

ion,

cop

ying

from

oth

er c

andi

date

s, u

sing

unau

thor

ized

mat

eria

ls su

ch a

s pr

epar

ed a

nsw

ers

and

note

s, c

ollu

sion

amon

g ca

ndid

ates

via

mob

ile p

hone

s, pa

ssin

g of

pap

er, o

r equ

ival

ent,

and

prov

ision

of e

xter

nal a

ssist

ance

via

the

supe

rviso

r, m

obile

pho

ne, e

tc. T

o re

duce

in

appr

opria

te b

ehav

ior,

an E

xam

inat

ion

Mon

itorin

g Ce

ll w

as e

stab

lishe

d w

ithin

the

Educ

atio

n De

part

men

t, w

hich

, in

addi

tion

to th

e m

onito

ring

done

by

the

Boar

ds th

emse

lves

and

the

mon

itorin

g by

retir

ed g

over

nmen

t offi

cial

s, h

as s

tren

gthe

ned

the

mon

itorin

g of

exa

ms.

Pre

ss c

onfe

renc

es a

re a

lso u

sed

to ra

ise

awar

enes

s abo

ut th

e co

nseq

uenc

es o

f ina

ppro

pria

te b

ehav

ior d

urin

g th

e ex

amin

atio

n.

20

. The

cre

dibi

lity

of t

he e

xam

s an

d its

res

ults

are

cal

led

into

que

stio

n w

ith r

egar

ds t

o th

e ne

ed f

or q

ualit

y im

prov

emen

t at

all

leve

ls (e

xam

des

ign,

ad

min

istra

tion,

scor

ing,

and

det

erm

inin

g re

sults

) and

the

perc

eptio

n of

the

gene

ral p

ublic

that

che

atin

g is

still

wid

espr

ead.

The

vie

ws o

f sta

keho

lder

s ten

d to

va

ry in

ter

ms

of t

heir

area

of c

once

rn. M

edia

rep

orts

ten

d to

focu

s on

the

con

duct

(i.e

., oc

curr

ence

of c

heat

ing)

and

acc

urac

y or

del

ays

in r

esul

ts, a

s do

pa

rent

s an

d st

uden

ts. T

his

year

, the

re h

as b

een

a su

bsta

ntia

l am

ount

of c

ritic

ism o

ver t

he c

ompu

teriz

atio

n of

the

proc

ess.

In a

dditi

on to

thes

e iss

ues,

sta

ff w

ithin

the

Boa

rds,

gov

ernm

ent

depa

rtm

ents

, an

d te

ache

rs a

ppea

rs t

o be

con

cern

ed w

ith i

ssue

s in

mar

king

. Fi

nally

, fr

om r

epor

ts a

nd i

nter

view

s, st

akeh

olde

rs s

uch

as th

ose

in te

ache

r edu

catio

n in

stitu

tions

, NGO

s, a

nd s

elec

ted

gove

rnm

ent o

ffici

als

and

educ

ator

s po

int o

ut th

e po

or q

ualit

y of

the

exam

pa

per a

nd it

s que

stio

ns. U

nive

rsiti

es d

o no

t app

ear t

o tr

ust t

he q

ualit

y of

the

exam

s and

hav

e th

eir o

wn

entr

ance

exa

ms.

21. R

espo

nden

ts h

ave

note

d th

at if

a s

tude

nt m

anag

es to

mak

e it

to s

econ

dary

sch

ool (

i.e.,

does

not

dro

p ou

t at e

arlie

r sta

ges

or fa

ce a

cces

s-re

late

d ba

rrie

rs),

then

fees

or

loca

tion

shou

ld n

ot s

erve

as

a ba

rrie

r to

tak

e th

e ex

amin

atio

ns a

s it

is th

e ke

y ce

rtifi

catio

n fo

r pr

ovin

g th

ey’v

e co

mpl

eted

sec

onda

ry s

choo

l. Ex

am c

ente

rs a

re u

sual

ly c

lose

to th

e st

uden

ts' s

choo

ls an

d th

ere

no re

gist

ratio

n ba

rrie

rs b

ased

on

ethn

icity

or g

ende

r. W

ith re

gard

s to

fees

, as o

f 201

0, th

e fe

es h

ave

been

wai

ved

by th

e Pu

njab

gov

ernm

ent,

stat

ing

that

fees

wer

e a

caus

e of

dro

pout

at s

econ

dary

leve

l. Fo

r th

e la

st tw

o ye

ars,

the

fees

hav

e no

t se

rved

as

a ba

rrie

r. W

hen

look

ing

at th

e nu

mbe

rs o

f stu

dent

s w

ho a

ppea

red

for t

he e

xam

s in

201

0, n

ews

repo

rts

estim

ate

abou

t 1.1

mill

ion

for a

ll Bo

ards

an

d th

is is

simila

r to

the

enro

llmen

t fig

ures

for G

rade

10

from

200

8 of

abo

ut 1

mill

ion

(PSL

M).

One

cou

ld a

ssum

e th

at e

ven

with

a p

opul

atio

n in

crea

se, t

he

num

ber i

s qui

te si

mila

r.

22

. Bec

ause

it is

unc

lear

wha

t th

e of

ficia

l pur

pose

s an

d us

es o

f exa

min

atio

n re

sults

are

, it

is di

fficu

lt to

iden

tify

"impr

oper

" us

es. G

ener

ally

, how

ever

, exa

m

resu

lts a

re n

ot u

sed

beyo

nd d

eter

min

ing

pass

/fai

l and

adm

issio

n in

to th

e ne

xt le

vel o

f edu

catio

n, w

hich

are

in li

ne w

ith th

e in

tend

ed p

urpo

ses.

23. S

tude

nts m

ay re

take

the

exam

inat

ion,

may

opt

for l

ess s

elec

tive

scho

ols/

univ

ersit

ies/

trac

ks, o

r the

y m

ay re

peat

the

grad

e.

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

32

PU

NJA

B PR

OVI

NCE

, PAK

ISTA

N Na

tiona

l (or

Sys

tem

-Lev

el) L

arge

-Sca

le A

sses

smen

t (NL

SA)

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

33

ENAB

LIN

G CO

NTE

XT

Ove

rall

fram

ewor

k of

pol

icie

s, le

ader

ship

, org

aniza

tiona

l str

uctu

res,

fisca

l and

hum

an re

sour

ces i

n w

hich

NLS

A ac

tivity

take

s pla

ce in

a co

untr

y or

syst

em a

nd th

e ex

tent

to w

hich

that

fram

ewor

k is

cond

uciv

e to

, or s

uppo

rtiv

e of

, the

NLS

A ac

tivity

. LA

TEN

T EM

ERGI

NG

ESTA

BLIS

HED

AD

VAN

CED

ENAB

LIN

G CO

NTE

XT 1

:

Sett

ing

clea

r po

licie

s for

NLS

A

No

NLS

A ex

erci

se h

as ta

ken

plac

e.

The

NLS

A ha

s be

en

oper

atin

g on

an

irr

egul

ar b

asis.

1

The

NLS

A is

a st

able

pro

gram

tha

t ha

s be

en o

pera

ting

regu

larly

. Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

Ther

e is

no p

olic

y do

cum

ent

pert

aini

ng

to N

LSA.

Th

ere

is an

in

form

al

or

draf

t po

licy

docu

men

t tha

t aut

horiz

es th

e N

LSA.

2

Ther

e is

a fo

rmal

pol

icy

docu

men

t th

at

auth

orize

s the

NLS

A.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

e po

licy

docu

men

t is

not

avai

labl

e to

th

e pu

blic

. Th

e po

licy

docu

men

t is

avai

labl

e to

the

pu

blic

. Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

Ther

e is

no p

lan

for N

LSA

activ

ity.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

ere

is a

gene

ral u

nder

stan

ding

that

the

NLS

A w

ill ta

ke p

lace

. 3

Ther

e is

a w

ritte

n N

LSA

plan

for

the

co

min

g ye

ars.

ENAB

LIN

G CO

NTE

XT 2

: H

avin

g st

rong

pub

lic e

ngag

emen

t for

NLS

A Al

l st

akeh

olde

r gr

oups

str

ongl

y op

pose

th

e N

LSA

or a

re in

diffe

rent

to it

. 4

Som

e st

akeh

olde

r gr

oups

op

pose

th

e N

LSA.

M

ost

stak

ehol

ders

gro

ups

supp

ort

the

NLS

A.

All s

take

hold

er g

roup

s sup

port

the

NLS

A.

(CO

NTI

NU

ED)

* * *

*

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

34

LA

TEN

T EM

ERGI

NG

ESTA

BLIS

HED

AD

VAN

CED

ENAB

LIN

G CO

NTE

XT 3

: H

avin

g re

gula

r fu

ndin

g fo

r NL

SA

Ther

e is

no f

undi

ng a

lloca

ted

to t

he

NLS

A.

Ther

e is

irreg

ular

fun

ding

allo

cate

d to

th

e N

LSA.

Th

ere

is re

gula

r fu

ndin

g al

loca

ted

to t

he

NLS

A. 5

Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Fu

ndin

g co

vers

so

me

core

N

LSA

activ

ities

: des

ign,

adm

inist

ratio

n, a

naly

sis

and

repo

rtin

g.

Fund

ing

cove

rs a

ll co

re N

LSA

activ

ities

: de

sign,

ad

min

istra

tion,

an

alys

is an

d re

port

ing.

6

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Fu

ndin

g do

es n

ot c

over

res

earc

h an

d de

velo

pmen

t act

iviti

es.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Fu

ndin

g co

vers

re

sear

ch

and

deve

lopm

ent a

ctiv

ities

.

ENAB

LIN

G CO

NTE

XT 4

: H

avin

g st

rong

org

aniz

atio

nal s

truc

ture

s for

NLS

A

Ther

e is

no N

LSA

offic

e, a

d ho

c un

it or

te

am.

The

NLS

A of

fice

is a

tem

pora

ry a

genc

y or

gr

oup

of p

eopl

e.

The

NLS

A of

fice

is a

perm

anen

t ag

ency

, in

stitu

tion

or u

nit. 7

Th

is op

tion

does

no

t ap

ply

to

this

di

men

sion.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Po

litic

al c

onsid

erat

ions

reg

ular

ly h

ampe

r te

chni

cal c

onsid

erat

ions

. Po

litic

al

cons

ider

atio

ns

som

etim

es

ham

per t

echn

ical

con

sider

atio

ns.

Polit

ical

co

nsid

erat

ions

ne

ver

ham

per

tech

nica

l con

sider

atio

ns.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

e N

LSA

offic

e is

not

acco

unta

ble

to a

cl

early

reco

gniz

ed b

ody.

Th

e N

LSA

offic

e is

acco

unta

ble

to

a cl

early

reco

gniz

ed b

ody.

8

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

(CO

NTI

NU

ED)

*

*

*

*

**

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

35

LATE

NT

EMER

GIN

G ES

TABL

ISH

ED

ADVA

NCE

D

ENAB

LIN

G CO

NTE

XT 5

:

Hav

ing

effe

ctiv

e hu

man

res

ourc

es fo

r NL

SA

Ther

e is

no s

taff

allo

cate

d fo

r ru

nnin

g an

N

LSA.

Th

e N

LSA

offic

e is

inad

equa

tely

sta

ffed

to e

ffect

ivel

y ca

rry

out t

he a

sses

smen

t. 9

Th

e N

LSA

offic

e is

adeq

uate

ly s

taffe

d to

ca

rry

out

the

NLS

A ef

fect

ivel

y,

with

m

inim

al is

sues

.

The

NLS

A of

fice

is ad

equa

tely

sta

ffed

to

carr

y ou

t th

e N

LSA

effe

ctiv

ely,

with

no

issue

s.

The

coun

try/

syst

em

does

no

t of

fer

oppo

rtun

ities

th

at

prep

are

indi

vidu

als

for w

ork

on N

LSA.

10

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

e co

untr

y/sy

stem

of

fers

so

me

oppo

rtun

ities

to

prep

are

indi

vidu

als

for

wor

k on

the

NLS

A.

The

coun

try/

syst

em o

ffers

a w

ide

rang

e of

opp

ortu

nitie

s to

pre

pare

ind

ivid

uals

for w

ork

on th

e N

LSA.

*

*

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

36

SYST

EM A

LIGN

MEN

T De

gree

to w

hich

the

NLSA

is c

oher

ent w

ith o

ther

com

pone

nts o

f the

edu

catio

n sy

stem

. LA

TEN

T EM

ERGI

NG

ESTA

BLIS

HED

AD

VAN

CED

SYST

EM A

LIGN

MEN

T 1:

Al

igni

ng th

e NL

SA w

ith le

arni

ng g

oals

It is

not

clea

r if

the

NLS

A is

base

d on

cu

rric

ulum

or l

earn

ing

stan

dard

s.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

e N

LSA

mea

sure

s pe

rfor

man

ce a

gain

st

curr

icul

um o

r lea

rnin

g st

anda

rds.

Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

Wha

t th

e N

LSA

mea

sure

s is

gene

rally

qu

estio

ned

by st

akeh

olde

r gro

ups.

Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

Wha

t th

e N

LSA

mea

sure

s is

ques

tione

d by

som

e st

akeh

olde

r gro

ups.

11

Wha

t th

e N

LSA

mea

sure

s is

larg

ely

acce

pted

by

stak

ehol

der g

roup

s.

Ther

e ar

e no

mec

hani

sms

in p

lace

to

ensu

re

that

th

e N

LSA

accu

rate

ly

mea

sure

s w

hat

it is

supp

osed

to

m

easu

re.

Ther

e ar

e ad

hoc

rev

iew

s of

the

NLS

A to

en

sure

th

at

it m

easu

res

wha

t it

is in

tend

ed to

mea

sure

.

Ther

e ar

e re

gula

r in

tern

al r

evie

ws

of th

e N

LSA

to e

nsur

e th

at it

mea

sure

s w

hat

it is

inte

nded

to m

easu

re. 1

2

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

SYST

EM A

LIGN

MEN

T 2:

Pr

ovid

ing

teac

hers

wit

h op

port

uniti

es to

lear

n ab

out t

he N

LSA

Ther

e ar

e no

cou

rses

or

wor

ksho

ps o

n th

e N

LSA.

Th

ere

are

occa

siona

l co

urse

s or

w

orks

hops

on

the

NLS

A.

Ther

e ar

e so

me

cour

ses

or w

orks

hops

on

the

NLS

A of

fere

d on

a re

gula

r bas

is.

Ther

e ar

e w

idel

y av

aila

ble

high

qua

lity

cour

ses

or

wor

ksho

ps

on

the

NLS

A of

fere

d on

a re

gula

r bas

is.

* * *

*

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

37

ASSE

SSM

ENT

QU

ALIT

Y De

gree

to w

hich

the

NLSA

mee

ts te

chni

cal s

tand

ards

, is f

air,

and

is us

ed in

an

effe

ctiv

e w

ay.

LATE

NT

EMER

GIN

G ES

TABL

ISH

ED

ADVA

NCE

D

ASSE

SSM

ENT

QU

ALIT

Y 1:

En

suri

ng th

e qu

alit

y of

the

NLSA

No

optio

ns a

re o

ffere

d to

inc

lude

all

grou

ps o

f stu

dent

s in

the

NLS

A.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

At

leas

t on

e op

tion

is of

fere

d to

incl

ude

all g

roup

s of s

tude

nts i

n th

e N

LSA.

13

Diffe

rent

opt

ions

are

offe

red

to i

nclu

de

all g

roup

s of s

tude

nts i

n th

e N

LSA.

Ther

e ar

e no

mec

hani

sms

in p

lace

to

ensu

re th

e qu

ality

of t

he N

LSA.

Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

Ther

e ar

e so

me

mec

hani

sms

in p

lace

to

ensu

re th

e qu

ality

of t

he N

LSA.

Th

ere

are

a va

riety

of

mec

hani

sms

in

plac

e to

ens

ure

the

qual

ity o

f the

NLS

A. 1

4

Ther

e is

no t

echn

ical

rep

ort

or o

ther

do

cum

enta

tion

abou

t the

NLS

A.

Ther

e is

som

e do

cum

enta

tion

abou

t th

e te

chni

cal

aspe

cts

of t

he N

LSA,

but

it

is no

t in

a fo

rmal

repo

rt fo

rmat

. 15

Ther

e is

a co

mpr

ehen

sive

tech

nica

l re

port

but

with

rest

ricte

d ci

rcul

atio

n.

Ther

e is

a co

mpr

ehen

sive,

hig

h qu

ality

te

chni

cal r

epor

t av

aila

ble

to t

he g

ener

al

publ

ic.

ASSE

SSM

ENT

QU

ALIT

Y 2:

En

suri

ng e

ffect

ive

uses

of t

he N

LSA

NLS

A re

sults

are

not

diss

emin

ated

. N

LSA

resu

lts a

re p

oorly

diss

emin

ated

. 16

NLS

A re

sults

ar

e di

ssem

inat

ed

in

an

effe

ctiv

e w

ay.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

NLS

A in

form

atio

n is

not

used

or

is us

ed

in w

ays

inco

nsist

ent

with

the

pur

pose

s or

the

tec

hnic

al c

hara

cter

istic

s of

the

as

sess

men

t. 17

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

N

LSA

resu

lts

are

used

by

so

me

stak

ehol

der

grou

ps

in

a w

ay

that

is

cons

isten

t w

ith

the

purp

oses

an

d te

chni

cal

char

acte

ristic

s of

th

e as

sess

men

t.

NLS

A in

form

atio

n is

used

by

al

l st

akeh

olde

r gr

oups

in

a

way

th

at

is co

nsist

ent

with

th

e pu

rpos

es

and

tech

nica

l ch

arac

teris

tics

of

the

asse

ssm

ent.

Ther

e ar

e no

mec

hani

sms

in p

lace

to

mon

itor

the

cons

eque

nces

of

the

NLS

A.

18

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

ere

are

som

e m

echa

nism

s in

pla

ce t

o m

onito

r the

con

sequ

ence

s of t

he N

LSA.

Th

ere

are

a va

riety

of

mec

hani

sms

in

plac

e to

mon

itor

the

cons

eque

nces

of

the

NLS

A.

*

*

*

* *

*

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

38

Natio

nal (

or S

yste

m-L

evel

) Lar

ge S

cale

Ass

essm

ent (

NLSA

): Developmen

t-level rating j

ustifications

1.

The

Punj

ab E

duca

tion

Asse

ssm

ent S

yste

m (P

EAS)

was

adm

inist

ered

in 2

011

inde

pend

ently

. Prio

r to

that

, PEA

S w

as p

art o

f the

Nat

iona

l Ass

essm

ent (

Nat

iona

l Ed

ucat

ion

Asse

ssm

ent S

yste

m).

Und

er N

EAS,

the

prov

ince

impl

emen

ted

the

asse

ssm

ent t

o a

prov

inci

ally

repr

esen

tativ

e sa

mpl

e of

stu

dent

s an

d pu

blish

ed a

pr

ovin

cial

rep

ort s

epar

ate

from

the

natio

nal r

epor

t. Th

e Pu

njab

Exa

min

atio

n Co

mm

issio

n (P

EC) w

as e

stab

lishe

d in

200

6 an

d aw

arde

d a

stat

utor

y st

atus

in

2010

thro

ugh

an a

ct o

f par

liam

ent.

PEC

is re

spon

sible

for d

esig

ning

, im

plem

entin

g an

d re

port

ing

on a

sses

smen

ts fo

r all

Grad

e 5

(prim

ary

scho

ol) a

nd G

rade

8

(mid

dle

scho

ol) s

tude

nts

in P

unja

b. P

EC c

ondu

cts

Grad

e 5

and

Grad

e 8

annu

al e

xam

inat

ions

in a

ll su

bjec

ts fo

r al

l the

pub

lic a

nd p

rivat

e sc

hool

s in

Pun

jab.

PE

C pr

ovid

es in

divi

dual

stu

dent

rep

orts

tha

t gr

ade

the

stud

ent’s

per

form

ance

on

each

of t

he e

xpec

ted

stud

ent

lear

ning

out

com

es (S

LOs)

by

subj

ect.

The

mai

n ob

ject

ive

of th

ese

exam

inat

ions

is to

dev

elop

a k

ey d

atab

ase

abou

t chi

ldre

n’s

perf

orm

ance

and

, as

a re

sult,

mea

sure

the

perf

orm

ance

of t

he te

ache

rs

and

scho

ols.

2.

N

otifi

catio

n of

Pun

jab

Educ

atio

n As

sess

men

t Sys

tem

(PEA

S) c

over

s th

e pe

riod

2005

-200

8, is

sued

in 2

005

by G

over

nmen

t of P

unja

b, E

duca

tion

Depa

rtm

ent

(whi

le P

EAS

was

stil

l a p

art

of N

EAS)

. Thi

s no

tific

atio

n ha

s no

t be

en u

pdat

ed to

incl

ude

the

curr

ent

perio

d. T

his

is an

issu

e po

inte

d ou

t by

sta

ff m

embe

rs

them

selv

es, w

ho re

cogn

ize th

at it

nee

ds to

be

raise

d w

ith th

e Se

cret

ary,

but

no

timel

ine

was

pro

vide

d fo

r whe

n th

is w

ill h

appe

n.

3.

Th

e pl

an fo

r the

nex

t tw

o ye

ars

has

been

dev

elop

ed. T

he w

ritte

n pl

an, u

noffi

cial

ly a

ppro

ved

by th

e Se

cret

ary

of E

duca

tion,

is c

urre

ntly

und

er d

evel

opm

ent,

and

they

exp

ect t

o re

ceiv

e of

ficia

l app

rova

l in

the

next

few

mon

ths.

4.

Th

ere

does

not

app

ear t

o be

any

pub

licly

ava

ilabl

e ev

iden

ce fo

r sta

keho

lder

per

cept

ions

of t

he la

rge-

scal

e as

sess

men

ts.

5.

Th

ere

is re

gula

r fun

ding

allo

cate

d by

the

gove

rnm

ent,

as w

ell a

s tec

hnic

al a

ssist

ance

from

the

Wor

ld B

ank

and

DFID

.

6.

PEAS

rec

eive

s sin

gle-

line

fund

ing

that

is n

ot a

lloca

ted

to a

ny o

ne a

ctiv

ity, t

hen

it se

eks

appr

oval

for

pla

nned

act

iviti

es. P

lann

ing

does

not

app

ear

as a

se

para

te a

ctiv

ity in

the

wor

k pl

an, h

owev

er p

lann

ing

occu

rs tw

ice

a ye

ar a

s pe

r go

vern

men

t ru

les,

in w

hich

fund

ing

can

be r

eallo

cate

d to

exi

stin

g or

new

ac

tiviti

es.

7.

Th

e N

LSA

offic

e is

a pe

rman

ent

inst

itutio

n cr

eate

d fo

r ru

nnin

g th

e N

LSA.

It w

as f

irst

esta

blish

ed in

200

2 as

the

Pro

vinc

ial E

duca

tion

Asse

ssm

ent

Cent

re

(PEA

CE),

Punj

ab, a

nd in

200

5 it

was

rena

med

to th

e Pu

njab

Edu

catio

n As

sess

men

t Sys

tem

(PEA

S).

8.

It

is ac

coun

tabl

e to

the

Punj

ab D

epar

tmen

t of E

duca

tion.

9.

Ther

e is

perm

anen

t or f

ull-t

ime

staf

f, bu

t it i

s ins

uffic

ient

to m

eet t

he n

eeds

of t

he a

sses

smen

t. Is

sues

that

hav

e be

en id

entif

ied

with

the

perf

orm

ance

of t

he

hum

an re

sour

ces

that

are

resp

onsib

le fo

r the

NLS

A in

clud

e w

eakn

esse

s in

test

des

ign,

freq

uent

err

ors

in d

ata

proc

essin

g, a

nd d

elay

s du

e to

dep

ende

nce

on

exte

rnal

hum

an re

sour

ces

for,

for e

xam

ple,

pap

er d

evel

opm

ent,

data

ent

ry, a

nd a

naly

sis. W

ith re

gard

to is

sues

with

item

dev

elop

men

t, it

has

been

diff

icul

t to

find

rel

evan

t/go

od q

ualit

y sp

ecia

lists

to

deve

lop

item

s, e

spec

ially

the

kin

d of

item

s th

at g

o be

yond

the

tex

tboo

k an

d th

at a

sses

s hi

gher

ord

er t

hink

ing

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

39

skill

s. T

here

is a

lso n

o in

-hou

se c

apac

ity to

dev

elop

the

sam

ple.

Add

ition

ally

, in-

hous

e st

aff l

ack

the

requ

ired

skill

s to

ben

efit

from

the

tech

nica

l ass

istan

ce

that

was

pro

vide

d. E

ffect

ive

data

man

agem

ent a

nd st

orag

e is

also

an

issue

.

10. I

n th

e pa

st, t

he U

nive

rsity

of E

duca

tion

(Lah

ore)

offe

red

shor

t cou

rses

, and

a s

peci

alize

d co

urse

was

offe

red

by th

e U

nive

rsity

of M

elbo

urne

in c

olla

bora

tion

with

the

Punj

ab U

nive

rsity

Inst

itute

of E

duca

tiona

l Res

earc

h (IE

R) fo

r NEA

S st

aff.

How

ever

non

e of

thes

e ha

ve b

een

sust

aine

d.

11

. PEA

S st

aff n

ote

that

ther

e ha

s be

en s

ome

conc

ern,

par

ticul

arly

from

teac

hers

, ove

r w

hat t

he a

sses

smen

t cov

ers

beca

use

they

are

use

d to

text

book

-bas

ed

exam

s and

an

asse

ssm

ent t

hat l

inks

itse

lf to

the

curr

icul

a an

d as

ks q

uest

ions

in a

diff

eren

t man

ner i

s unf

amili

ar to

them

(tex

tboo

ks c

anno

t be

said

to b

e w

ell

alig

ned

with

the

curr

ent c

urric

ula

as th

ey a

re n

ot o

nly

base

d on

an

olde

r ver

sion

of th

e cu

rric

ula,

dev

elop

ed in

200

2, b

ut th

ey a

lso d

o no

t ade

quat

ely

cove

r al

l asp

ects

of t

he c

urric

ula

such

as

high

er o

rder

thin

king

ski

lls).

How

ever

, thi

s se

ntim

ent h

as d

ecre

ased

ove

r the

yea

rs w

ith te

ache

rs’ i

ncre

ased

fam

iliar

ity o

f th

e as

sess

men

t and

thei

r rea

lizin

g th

at it

is n

ot a

hig

h st

akes

test

.

12. F

or t

he P

EAS

2011

ass

essm

ent,

ther

e ha

ve b

een

seve

ral

revi

ews

at d

iffer

ent

stag

es,

durin

g th

e de

velo

pmen

t of

the

ass

essm

ent

fram

ewor

k, i

tem

de

velo

pmen

t and

afte

r the

pilo

t. Th

ese

are

inte

rnal

revi

ews

that

incl

ude

exte

rnal

exp

erts

from

the

univ

ersit

ies,

teac

her p

rofe

ssio

nal d

evel

opm

ent c

olle

ges,

and

scho

ols.

13. T

he N

LSA

is of

fere

d in

the

lang

uage

of i

nstr

uctio

n fo

r alm

ost a

ll st

uden

t gro

ups a

nd in

var

ious

geo

grap

hica

l reg

ions

of P

unja

b.

14

. A v

arie

ty o

f mec

hani

sms

are

in p

lace

to

ensu

re t

he q

ualit

y of

the

NLS

A in

stru

men

t. Sp

ecifi

cally

, all

proc

tors

or

adm

inist

rato

rs a

re t

rain

ed a

ccor

ding

to

a pr

otoc

ol, a

nd t

here

is a

sta

ndar

dize

d m

anua

l for

larg

e-sc

ale

asse

ssm

ent

adm

inist

rato

rs. A

dditi

onal

ly, a

pilo

t is

cond

ucte

d be

fore

the

mai

n da

ta c

olle

ctio

n ta

kes

plac

e an

d al

l boo

klet

s ar

e nu

mbe

red.

The

re is

also

inte

rnal

and

ext

erna

l rev

iew

by

obse

rver

s. W

ith r

egar

d to

sco

ring,

the

re is

a t

wo-

day

trai

ning

of

scor

ers,

pro

visio

n of

det

aile

d co

debo

oks

and

rubr

ics

for o

pen-

ende

d ite

ms,

and

10-1

5 pe

rcen

t che

ck o

f sco

ring

by a

noth

er p

erso

n. In

term

s of

pro

cess

ing

of

data

, the

re a

re p

roto

cols

for e

nsur

ing

the

qual

ity o

f dat

a en

try,

10

perc

ent d

ata

chec

k an

d cl

eani

ng. T

here

doe

s not

app

ear t

o be

dou

ble

proc

essin

g of

dat

a.

15

. The

tech

nica

l rep

ort f

or P

EAS

has b

een

deve

lope

d an

d is

curr

ently

bei

ng p

rinte

d. F

or N

EAS

ther

e w

ere

regu

lar t

echn

ical

repo

rts b

ut w

ith li

mite

d ci

rcul

atio

n.

16

. The

PEA

S 20

11 A

sses

smen

t res

ults

and

repo

rts

have

not

bee

n fin

alize

d an

d th

us th

ere

has

been

no

diss

emin

atio

n at

this

stag

e. P

lans

incl

ude

thre

e ty

pes

of

repo

rts

(tech

nica

l, th

emat

ic f

or d

iffer

ent

audi

ence

s, a

nd s

impl

ified

for

a g

ener

al a

udie

nce)

, as

wel

l as

dist

rict-

base

d w

orks

hops

. Und

er N

EAS

ther

e w

ere

year

ly re

port

s an

d ne

wsle

tter

s, s

take

hold

er c

onfe

renc

es a

nd c

over

age

in th

e m

edia

. How

ever

, diss

emin

atio

n w

as n

oted

as

one

of th

e w

eake

st a

reas

in th

e en

tire

prog

ram

by

a fo

rmer

sta

ff m

embe

r. Du

e to

the

lack

of c

apac

ity a

nd d

edic

ated

sta

ff th

roug

hout

mos

t of t

he p

roje

ct, d

evel

opin

g re

port

s of

ten

took

a

grea

t dea

l of t

ime

and

caus

ed d

elay

s in

diss

emin

atio

n.

17

. Giv

en th

at th

e cu

rren

t PEA

S as

sess

men

t res

ults

hav

e ye

t to

be d

eter

min

ed it

is n

ot p

ossib

le to

say

how

the

info

rmat

ion

from

this

part

icul

ar a

sses

smen

t will

be

use

d. In

pre

viou

s ex

perie

nces

, the

re h

as b

een

som

e, a

lthou

gh s

pora

dic,

use

of r

esul

ts. F

or e

xam

ple

the

Nat

iona

l Edu

catio

n Po

licy

2009

men

tions

NEA

S re

sults

, and

the

Dire

ctor

ate

of S

taff

Deve

lopm

ent (

DSD)

in P

unja

b cl

aim

s to

have

use

d re

sults

in tr

aini

ng. H

owev

er, f

or th

e m

ost p

art v

ery

little

use

is fo

und,

an

d im

plem

enta

tion

of th

e as

sess

men

t rec

omm

enda

tions

has

bee

n al

mos

t non

-exi

sten

t. Th

is is

due

to la

ck o

f a m

echa

nism

for e

nsur

ing

the

use

of re

sults

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

40

and

to s

ome

exte

nt la

ck o

f m

otiv

atio

n to

ens

ure

impl

emen

tatio

n on

the

par

t of

pol

icym

aker

s. It

also

has

par

tly t

o do

with

the

lack

of

a re

sear

ch-b

ased

de

cisio

n m

akin

g pr

actic

es a

nd u

nder

stan

ding

of t

he u

ses o

f ass

essm

ent r

esul

ts.

18

. The

nee

d is

reco

gnize

d by

PEA

S an

d th

is m

ay h

appe

n in

the

futu

re p

rovi

ded

ther

e is

fund

ing.

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

41

PU

NJA

B PR

OVI

NCE

, PAK

ISTA

N In

tern

atio

nal L

arge

-Sca

le A

sses

smen

t (IL

SA)

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

42

ENAB

LIN

G CO

NTE

XT

Ove

rall

fram

ewor

k of

pol

icie

s, le

ader

ship

, org

aniza

tiona

l str

uctu

res,

fisca

l and

hum

an re

sour

ces i

n w

hich

ILSA

take

s pla

ce in

a c

ount

ry o

r sys

tem

and

the

exte

nt to

w

hich

that

fram

ewor

k is

cond

uciv

e to

, or s

uppo

rtiv

e of

, ILS

A ac

tivity

. LA

TEN

T EM

ERGI

NG

ESTA

BLIS

HED

AD

VAN

CED

ENAB

LIN

G CO

NTE

XT 1

:

Sett

ing

clea

r po

licie

s for

ILSA

The

coun

try/

syst

em h

as n

ot p

artic

ipat

ed

in a

n IL

SA in

the

last

10

year

s.1

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

e co

untr

y/sy

stem

has

par

ticip

ated

in

at le

ast o

ne IL

SA in

the

last

10

year

s.

The

coun

try/

syst

em h

as p

artic

ipat

ed i

n tw

o or

mor

e IL

SA in

the

last

10

year

s.

The

coun

try/

syst

em

has

not

take

n co

ncre

te st

eps t

o pa

rtic

ipat

e in

an

ILSA

in

the

next

5 y

ears

.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

e co

untr

y/sy

stem

has

tak

en c

oncr

ete

step

s to

part

icip

ate

in a

t lea

st o

ne IL

SA in

th

e ne

xt 5

yea

rs.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Ther

e is

no

polic

y do

cum

ent

that

ad

dres

ses p

artic

ipat

ion

in IL

SA.

Ther

e is

an

info

rmal

or

dr

aft

polic

y do

cum

ent t

hat a

ddre

sses

par

ticip

atio

n in

IL

SA.

Ther

e is

a fo

rmal

pol

icy

docu

men

t th

at

addr

esse

s par

ticip

atio

n in

ILSA

. Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

e po

licy

docu

men

t is

not

avai

labl

e to

th

e pu

blic

. Th

e po

licy

docu

men

t is

avai

labl

e to

the

pu

blic

. Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

ENAB

LIN

G CO

NTE

XT 2

: H

avin

g re

gula

r fu

ndin

g fo

r IL

SA

Ther

e is

no f

undi

ng f

or p

artic

ipat

ion

in

ILSA

. Th

ere

is fu

ndin

g fr

om lo

ans

or e

xter

nal

dono

rs.

Ther

e is

regu

lar

fund

ing

allo

cate

d at

di

scre

tion.

Th

ere

is re

gula

r fun

ding

app

rove

d by

law

, de

cree

or n

orm

.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Fu

ndin

g co

vers

som

e co

re a

ctiv

ities

of

the

ILSA

. Fu

ndin

g co

vers

all

core

act

iviti

es o

f th

e IL

SA.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Fund

ing

does

not

cov

er r

esea

rch

and

deve

lop m

ent a

ctiv

ities

. Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Fu

ndin

g co

vers

re

sear

ch

and

deve

lopm

ent a

ctiv

ities

.

(CO

NTI

NU

ED)

* *

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

43

LATE

NT

EMER

GIN

G ES

TABL

ISH

ED

ADVA

NCE

D

ENAB

LIN

G CO

NTE

XT 3

: H

avin

g ef

fect

ive

hum

an r

esou

rces

for

ILSA

Th

ere

is no

te

am

or

natio

nal/s

yste

m

coor

dina

tor

to

carr

y ou

t th

e IL

SA

activ

ities

.

Ther

e is

a te

am

or

natio

nal/s

yste

m

coor

dina

tor

to

carr

y ou

t th

e IL

SA

activ

ities

.

Ther

e is

a te

am

and

natio

nal/s

yste

m

coor

dina

tor

to

carr

y ou

t th

e IL

SA

activ

ities

.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

e na

tiona

l/sys

tem

co

ordi

nato

r or

ot

her

desig

nate

d te

am m

embe

r m

ay n

ot

be

fluen

t in

th

e la

ngua

ge

of

the

asse

ssm

ent.

The

natio

nal/s

yste

m c

oord

inat

or is

flue

nt

in th

e la

ngua

ge o

f the

ass

essm

ent.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

e IL

SA o

ffice

is in

adeq

uate

ly s

taffe

d or

tr

aine

d to

ca

rry

out

the

asse

ssm

ent

effe

ctiv

ely.

The

ILSA

offi

ce i

s ad

equa

tely

sta

ffed

or

trai

ned

to c

arry

out

the

ILSA

effe

ctiv

ely,

w

ith m

inim

al is

sues

.

The

ILSA

offi

ce is

ade

quat

ely

staf

fed

and

trai

ned

to c

arry

out

the

ILSA

effe

ctiv

ely,

w

ith n

o iss

ues.

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

44

SYST

EM A

LIGN

MEN

T De

gree

to w

hich

the

ILSA

mee

ts te

chni

cal q

ualit

y st

anda

rds,

is fa

ir, a

nd is

use

d in

an

effe

ctiv

e w

ay.

LATE

NT

EMER

GIN

G ES

TABL

ISH

ED

ADVA

NCE

D

SYST

EM A

LIGN

MEN

T 1:

Pr

ovid

ing

oppo

rtun

ities

to le

arn

abou

t ILS

A Th

e IL

SA

team

ha

s no

t at

tend

ed

inte

rnat

iona

l wor

ksho

ps o

r mee

tings

. Th

e IL

SA

team

at

tend

ed

som

e in

tern

atio

nal w

orks

hops

or m

eetin

gs.

The

ILSA

tea

m a

tten

ded

all i

nter

natio

nal

wor

ksho

ps o

r mee

tings

. Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

The

coun

try/

syst

em

offe

rs

no

oppo

rtun

ities

to le

arn

abou

t ILS

A.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

e co

untr

y/sy

stem

of

fers

so

me

oppo

rtun

ities

to le

arn

abou

t ILS

A.

The

coun

try/

syst

em o

ffers

a w

ide

rang

e of

opp

ortu

nitie

s to

lear

n ab

out I

LSA.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

Opp

ortu

nitie

s to

lea

rn a

bout

ILS

A ar

e av

aila

ble

to t

he c

ount

ry's/

syst

em's

ILSA

te

am m

embe

rs o

nly.

Opp

ortu

nitie

s to

lea

rn a

bout

ILS

A ar

e av

aila

ble

to a

wid

e au

dien

ce, i

n ad

ditio

n to

th

e co

untr

y's/

syst

em's

ILSA

te

am

mem

bers

.

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

45

ASSE

SSM

ENT

QU

ALIT

Y De

gree

to w

hich

the

ILSA

mee

ts te

chni

cal q

ualit

y st

anda

rds,

is fa

ir, a

nd is

use

d in

an

effe

ctiv

e w

ay.

LATE

NT

EMER

GIN

G ES

TABL

ISH

ED

ADVA

NCE

D

ASSE

SSM

ENT

QU

ALIT

Y 1:

En

suri

ng th

e qu

alit

y of

ILSA

Da

ta

from

th

e IL

SA

has

not

been

pu

blish

ed.

The

coun

try/

syst

em

met

su

ffici

ent

stan

dard

s to

hav

e its

dat

a pr

esen

ted

bene

ath

the

mai

n di

spla

y of

th

e in

tern

atio

nal r

epor

t or i

n an

ann

ex.

The

coun

try/

syst

em

met

al

l te

chni

cal

stan

dard

s re

quire

d to

ha

ve

its

data

pr

esen

ted

in t

he m

ain

disp

lays

of

the

inte

rnat

iona

l rep

ort.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

The

coun

try/

syst

em h

as n

ot c

ontr

ibut

ed

new

kno

wle

dge

on IL

SA.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

The

coun

try/

syst

em h

as c

ontr

ibut

ed n

ew

know

ledg

e on

ILSA

.

ASSE

SSM

ENT

QU

ALIT

Y 2:

En

suri

ng e

ffect

ive

uses

of I

LSA

If an

y,

coun

try/

syst

em-s

peci

fic

resu

lts

and

info

rmat

ion

are

not

diss

emin

ated

in

the

coun

try/

syst

em.

Coun

try/

syst

em-s

peci

fic

resu

lts

and

info

rmat

ion

are

diss

emin

ated

irre

gula

rly

in th

e co

untr

y/sy

stem

.

Coun

try/

syst

em-s

peci

fic

resu

lts

and

info

rmat

ion

are

regu

larly

diss

emin

ated

in

the

coun

try/

syst

em.

Coun

try/

syst

em-s

peci

fic

resu

lts

and

info

rmat

ion

are

regu

larly

an

d w

idel

y di

ssem

inat

ed in

the

coun

try/

syst

em.

Prod

ucts

to

prov

ide

feed

back

to

scho

ols

and

educ

ator

s ab

out t

he IL

SA r

esul

ts a

re

not m

ade

avai

labl

e.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Pr

oduc

ts t

o pr

ovid

e fe

edba

ck t

o sc

hool

s an

d ed

ucat

ors

abou

t the

ILSA

res

ults

are

so

met

imes

mad

e av

aila

ble.

Prod

ucts

to

prov

ide

feed

back

to

scho

ols

and

educ

ator

s ab

out

ILSA

res

ults

ar

e sy

stem

atic

ally

mad

e av

aila

ble.

Ther

e is

no m

edia

cov

erag

e of

the

ILS

A re

sults

. Th

ere

is lim

ited

med

ia c

over

age

of t

he

ILSA

resu

lts.

Ther

e is

som

e m

edia

cov

erag

e of

the

IL

SA re

sults

. Th

ere

is w

ide

med

ia c

over

age

of th

e IL

SA

resu

lts.

If an

y,

coun

try/

syst

em-s

peci

fic

resu

lts

and

info

rmat

ion

from

the

ILS

A ar

e no

t us

ed t

o in

form

dec

ision

mak

ing

in t

he

coun

try/

syst

em.

Resu

lts f

rom

the

ILS

A ar

e us

ed i

n a

limite

d w

ay to

info

rm d

ecisi

on m

akin

g in

th

e co

untr

y/sy

stem

.

Resu

lts f

rom

the

ILSA

are

use

d in

som

e w

ays

to i

nfor

m d

ecisi

on m

akin

g in

the

co

untr

y/sy

stem

.

Resu

lts

from

th

e IL

SA

are

used

in

a

varie

ty

of

way

s to

in

form

de

cisio

n m

akin

g in

the

coun

try/

syst

em.

It is

not

clea

r th

at d

ecisi

ons

base

d on

IL

SA r

esul

ts h

ave

had

a po

sitiv

e im

pact

on

stud

ents

' ach

ieve

men

t lev

els.

This

optio

n do

es

not

appl

y to

th

is di

men

sion.

Th

is op

tion

does

no

t ap

ply

to

this

dim

ensio

n.

Deci

sions

bas

ed o

n th

e IL

SA r

esul

ts h

ave

had

a po

sitiv

e im

pact

on

st

uden

ts'

achi

evem

ent l

evel

s.

PUN

JAB

PRO

VIN

CE, P

AKIS

TAN

ǀ SA

BER-

STU

DEN

T AS

SESS

MEN

T

SABE

R PR

OVI

NCE

REP

ORT

|20

12

SYST

EMS

APPR

OAC

H FO

R BE

TTER

EDU

CATI

ON

RES

ULT

S

46

Inte

rnat

iona

l Lar

ge S

cale

Ass

essm

ent (

ILSA

): Developmen

t-level rating j

ustifications

1. T

he P

unja

b pr

ovin

ce h

as n

ot p

artic

ipat

ed in

an

ILSA

, and

it d

oes n

ot h

ave

plan

s to

do so

in th

e ne

ar fu

ture

.

PUNJAB PROVINCE, PAKISTAN ǀ SABER-STUDENT ASSESSMENT SABER PROVINCE REPORT | 2012

SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 46

Acknowledgements This report was prepared by The World Bank SABER-Student Assessment team in collaboration with Umbreen Arif, World Bank Senior Education Specialist and Task Team Leader for education projects in Pakistan.

References Clarke, M. 2012. “What Matters Most for Student Assessment Systems: A Framework Paper.” READ/SABER Working Paper Series. Washington, D.C.: World Bank.

The Government of the Punjab Province. 2012. The Punjab Curriculum Authority Act 2012. Punjab province: The Government of the Punjab Province, Pakistan.

World Bank. 2012. Pakistan Country Indicator Data. Washington, DC: World Bank. Data retrieved from http://data.worldbank.org/ on March 10, 2014.

World Bank. 2012. Pakistan - Second Punjab Education Sector Project. Report No. 66291-PK. Washington D.C.: The World Bank.

PUNJAB PROVINCE, PAKISTAN ǀ SABER-STUDENT ASSESSMENT SABER PROVINCE REPORT | 2012

SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 46

The Systems Approach for Better Education Results (SABER) initiative produces comparative data and knowledge on education policies and institutions, with the aim of helping countries systematically strengthen their education systems. SABER evaluates the quality of education policies against evidence-based global standards, using new diagnostic tools and detailed policy data. The SABER country reports give all parties with a stake in educational results—from administrators, teachers, and parents to policymakers and business people—an accessible, objective snapshot showing how well the policies of their country’s education system are oriented toward ensuring that all children and youth learn. This report focuses specifically on policies in the area of student assessment.

This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries.

THE WORLD BANK

www.worldbank.org/education/saber