How many students are paying attention? (Senegal)
What can you learn from visits to schools and classrooms?
The revelation of the secret and mysterious causality chains
Helen AbadziSenior Education SpecialistFTI Secretariatc/o World BankMay 12, [email protected]
APEIE workshop
Inside inputs get converted into information
Project inputs
BuildingsTextbooksSalariesTrainingSchool grantsSchool plansCommunity management
classroomactivities
Information processing
Impact evidence
EnrollmentsDropoutRepetitionSkillsLearning outcomes
Data collection from the “black box” ?? !!
This session will present variables that can be observed
Useful for you personally As well as for school surveys
How well was a treatment implemented?Find out
• Physical inputs – Building, maintenance, furniture, consumables– Radios, lab equipment, maps?– Textbooks
• Training inputs– For teachers, principals, administrators
• Instructional delivery to students• Management committees• School budgets• School planning
Building condition may affect implementation of other components
Poor mixture of cement and sand
Structural problemsLack of maintenance
Structural defects in a school
Computers and labsHow often do students use them?
With what software?
Teacher and management training coursesAsk:Did teachers and principals receive the
training that had been financed?How many days? What did they do?
Lectures, activities?What can they remember from it?
Please ask for three items they recallDo they use the training information in
classroom or in school?Ask: How? Where?
School feedingNutritional supplements
• Do students in fact get fed what the government thinks?
• Do the schools get enough?– What kinds?
• How often are they delivered?
• When were they last delivered?
• Take a look at them, see expiration date
Management improvement componentsFind out:School grants & budgets
What did the schools buy? Did not buy? In what ways (if any) did the grant purchases increase the
amount of learning?Community management
How often do they meet? What % come? What is the added value of their presence?
Buildings, money, monitoring? Do they just approve what principal decides?
School planning What did school decide was missing? What did they do about it? Do you see deficiencies that the school staff did not find?
Do school management committees perform the expected functions?
Where are the textbooks?Look for themNowhere?In the students’ hands?In the classroom?In the library?How many students vs. textbooks in a class?
If textbooks are insufficient, they stay on the shelves
Class time spent in blackboard copying
Instructional delivery to students
How much actual teaching do you see?
Scheduling visits to schools
Sampling schools – important issuesOmitting certain schools from surveys
alters the sampleIf more than 5% of the schools are changed
Visit unannounced if possible !Perhaps not reveal to local authorities when
schools will be visited
Classroom observation experiencefor youTake a look inside classes if possible without
being seen (e.g. from window)What were they doing before you went in?
Sit down for 5-10 minutes Take video or audio evidenceTeachers may try to teach better in front of
you but may not know how!After a while everyone may forget about you.
Which variables are worth measuring from classrooms events?
Use as framework the rules of how people learn
Look at:Is time spent for instruction?Are all students engaged in learning tasks?Does teacher know the subject?Is teacher using methods that help
students remember?How many interact with the teacher
Look around the class and think:Can the students learn the expected subjects in this class?
How many of the enrolled students present?
How many interact with the teacher, participate?
How is the class time spent?Activities carried out?
Are students kept busy all the time? Are students processing information or just
waiting?
Problems identified in many classroomsInstruction may not be going onThey may be unable to see the blackboardMay spend much time copyingMay repeat without understandingThey may receive no feedbackBe unoccupied most of the time Grade 1-2 textbooks may not really teach the
poor how to read
How much of the time that governments pay for is used for instruction?
Class: When financing is converted to information
In a class hour, students must retain as much info as possible and recall it effortlessly when needed
Time use has been measured many timesStallings classroom snapshot
Only a small fraction of the time that a government budgets is converted into information
Class time as allotted by a government (e.g., 200 days, 1000 teaching hours)
Remaining after school closures (strikes, weather, teacher training, extra holidays)
Remaining after teacher absenteeism and tardiness
Remaining after student absenteeism
Class time devoted to any learning task
Learning time relevant to curriculum
Teacher off task (India)
Legal teacher absenteeism:Teacher’s book from rural HondurasHow many days did this class study?
Students unoccupied, waiting for the bell to ring (most of them illiterate)
Brazil – group work?
Brazil – group work?
Teacher absenteeism and time use in Maputo: Schools constantly in recess
To remember and use information, students must:Receive itPractice it, contemplate itHave prior knowledge on which to fit itRead fluently to learn from booksKnow the official language well!do math fluently to solve complex problems
High-quality schools: Offer activities that create complex cognitive networks
Students recite + read long texts+ manipulate + collect real-world samples + answer questions connecting various items + derive new conclusions from data + solve problems + practice for fluency + generalize into various circumstancesSpend time in “active learning”
Mexico – efforts to reclassify
Signs that the students will probably remember the informationGroup or even individual work with students
concentratedActivities of students reading, discussing
not merely verbally repeatingWriting material that is not simply copyingTeacher uses aids like flash cardsTeacher monitors individual student work
Gives feedback Teacher going towards the back,
addressing individual students who did not volunteer to answer a question
Good: Group work, engaged studentsbut time must be kept strictly
•
Poor-quality schools:may teach items in series
with few connections
The heroes of the revolution are…
The books of the bible are….
2x2=4, 2x3=6, 2x4=8, 2x5=10….
Students may just recite or listen…
The principles of constitutional law are…
How many are paying attention?One student recites, rest unoccupied
Low-quality schools High quality schools
Recife, Brazil: seatwork – teacher not monitoring
Student attendance and participation
Determine whether students will learn
Check:How many of the enrolled students are present?We all hear about large classes
If you go towards the end of the school year will you find them?
Students may enroll but attend rarely, particularly when they are illiterate or do not know the material.
Is there space in a classroom for everyone enrolled?There may only be enough space for the
regular studentsWhat excuses do you get?
Students falling behind are often absent• Bank documents say that classes are large…• 19 of 45 students in grade 4, outside Nampula,
Mozambique
Bangladesh – student absenteeism
Look at students’ notebooks
• How much is written?Too much: no textbooks
Just 2-3 pages? No classes or frequent absences
Check for “hidden dropout”:How many of the students are actually involved in a class?Ask the teacher: is s/he up to date with the
curriculum that must be taught?If so, observe how many students actually
follow teachers’ instructions, answer questions, seem involved
The teacher may say that s/he is up to date, but may be only teaching 3-4 studentsRest may be illiterate and uninvolved
“Hidden dropout?”:Teachers in Nepal interacting
mainly with the front of the class
Do teachers know sufficient material to teach?See material on the blackboard
Does it make sense?Example: Students who don’t know letter values cannot
learn them from others’ fast reading
Rural Mozambiquerura
What impresses you most about this scene?
Do these students discriminate among letters of the fuzzy blackboard from this distance?
Lack of textbooks translates into loss of time at all levels
• Due to a lack of knowledge and materials, teachers do very few activities – But they can follow a textbook
• Teacher boredom:How many of us would spend 20 years in blackboard transcription?
Dictation in Burkina FasoIt takes 30 seconds to read and 7+ minutes to
dictate this phrase:
“Les industries Francaises ont connu un développement considerable mais rencontrent des difficultés dans les ressources.. »
10tth grade math– private school
Can the students read fluently?
Reading often not taught specificallyMethods are often very inefficient
Informal way to find out how many are literateTake book, go 3-4 pages aheadAsk a student to readMark the start of 1 minute by your watch,
start a paragraph, mark end in one minuteTake photo or count words on the spot
Ask 2-3 simple questions about what they read
What is their approximate reading speed?
Can students read their notes to you?Illiterate students writing “art”
Short-term memoryCrucial for reading comprehension
Long-term memory
12 seconds at mostAbout 7 items4 pictures
Sound (OLE2)
35 words per minute
Quelle est la langue d’instruction?Les langues Africaines ont orthographie régulière Peuvent être automatisées dans quelques mois
La fluidité transfert aux autres langues qui ont la même écriture
Grade 1 textbooks – critical for subsequent years
How should efficient textbooks instruct in reading?
Burkina Faso grade 1 book
Are there dictionaries in local languages?
Activity at entry in class Brazil 2002
State Ceará Rio Grande de Norte
Alagoas Goias São Paulo
Copying 6 40%
6 33%
2 22%
8 (2 secondary) 17%
Text production 1 7%
3 17%
3 33%
4 28%
2 4%
Reading lesson in class
2 13%
1 6%
1 11%
1 2%
No activity 1 7%
5 (2 waiting for bell) 28%
1 11%
8 17% (some
teachers absent) Teacher busy with 2-3 students only
1 7%
1 11%
4 8%
Teacher teaching whole class
2 13%
1 11%
6 43%
10 22%
Group practice 1 7%
2 11%
3 21%
4 8%
Art and Play 1(scheduled activity)
7%
1 (play with letters) 6%
1 (play with letters)
11%
1 (drawing) 7%
8 (during subject matter)
17%
No. classes 15 18 9 14 45
No schools 8 6 7 5 8
Apparently instructional activities (some suboptimal)
34% 45% 48% 92% 40%
Sample form to record instructional data from school
visitsSchool Grade Student & teacher
attendance
What was the class doing at the moment of mission entry? (How likely is this activity to engage all students in the task?)
How many sampled students can read? How well? Comments
Teac
her
pres
ent?
No.
stud
ents
en
rolle
d
No.
stud
ents
pr
esen
t
No.
of t
extb
ooks
in
sigh
t
Teac
her
teac
hing
en
tire
clas
s
Indi
vidu
al o
r ch
oral
ans
wer
s no
. res
pond
ing
1-2
stud
ents
at
blac
kboa
rd
Stud
ents
doi
ng
seat
wor
k al
one
Stud
ents
wor
king
in
gro
ups
Cla
ss c
opyi
ng
from
bla
ck
Boar
d O
ther
teac
hing
ac
tivity
: art
, pla
y si
ngin
g, e
xam
, m
ultip
les
Teac
her
chat
ting
or d
iscip
linin
g st
uden
ts
N
o cl
ear
inst
ruct
iona
l ac
tivity
A
ppro
x %
of
stud
ents
off
task
Teac
her
estim
ate
of n
umbe
r or
%
1. C
ould
har
dly
read
lette
rs,
wor
ds, o
r no
t at
all
2.B
arel
y de
code
d,
flat v
oice
, no
com
preh
ensio
n
3. R
ead
abou
t a
wor
d pe
r se
c.,
som
e st
umbl
ing
4.
Rea
d flu
ently
w
ith a
ppro
pria
te
expr
essi
on
How to measure and quantify classroom variables?Need for summative scales, “low-inference”
instrumentsMany observation instruments exist
On the minute, Flanders, Virgilio, CLASSClassroom observation instruments
DifficultcomplexImperfect
Stallings modified classroom snapshot
Low inferenceEasier to communicate to less educated staffFor lower-level grades, deprived environments
Where time gets wasted
Stallings 5-minute interaction – to be updated?
Snapshot 10 I : I ndividual T: Teacher S: Small Gp
L: Large Gp E: Entire Gp
Number of Adults Present Number of Students Present MATERIALS
ACTIVITY None Books Notebook Chalk
board Manipul
ative Visual/ Computational Aids
Co-operative
Reading Aloud T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E
I nstruction/ Demonstration/ Lecture T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E Discussion T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E
Practice/ Drill T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E Kinesthetic/ Projects T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E
Written Assignments/ Seatwork T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E Copying T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E
Assignments T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E Other T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E I 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E Social I nteraction T 1 S L E I 1 S L E
Student Uninvolved I 1 S L E Being Disciplined T 1 S L E Classroom Management T 1 S L E I 1 S L E Adult Social I nteraction/ Adult Uninvolved T Adult Management T Teacher out of Room T
Training and future useClassroom snapshot takes 5 days trainingTrainees learn how to spot time and wastage
Instructional time loss as a monitoring indicatorWhat % of the time bought by governments for
students is used in learning activities?Ghana 39% Pernambuco (Brazil) 63%Dominican Republic 65%Tunisia 79%
What % of the curriculum covered each year?What % of objectives do students master?
What test scores should we expect?What % of students in a class interacted with a
teacher?
Instructional Time Use in the participating countries
Pernambuco (Brazil)
Ghana
Morocco
Tunisia
School Year (Days) 200 197 204 190 School Closures (Days) 4.79 3.17 1.38 5.15 Days after Closures 195.21 193.83 202.62 184.85 Teacher Absence (Days) 12.76 43.01 13.36 11.55 Teacher Delays 5.50 39.75 6.94 1.27 Early Class Dismissals 2.31 2.43 6.68 1.22 School Year (Days left after losses) 174.65 108.6 175.6 170.8 % Year available for teaching 87.3% 55.1% 86.1% 89.9% Engagement Rate in Interactive or Passive Classroom Tasks 72.1% 70.2% 82.6% 86.7% School Days Devoted to Learning 125.9 76.3 145.1 148.1 School Year % Spent Engaged in Learning Tasks 63.0% 38.7% 71.1% 77.9% Student off Task Rate 19.3% 21.1% 9.2% 9.9% Student Absence (Days) 7.82 9.04 4.30 3.35 Student Delays (Number of Times) 5.64 10.61 5.187 2.63
Percentage of Time Spent in Instructional Tasks
Instructional Strategy % of time
U.S Classroom Criteria
Brazil Ghana Morocco Tunisia
Interactive instruction 50% or more 52.4 59.9 62.8 61.2
Oral Reading 6.7 8.7 15.7 15.3
Teaching, Explanation 32.8 19.9 26.7 27.9
Discussion 6.3 24.1 6.6 6.2
Practice Drill 1.4 6.5 12.3 11.3
Passive instruction 35% or less 19.6 10.3 19.9 25.6
Seatwork 16.3 7.4 14.8 22.9
Copy 3.0 2.9 5.0 2.7
Total Instructional Time 72.1 70.2 82.6 86.7
Organizing/Management 15% or less 27.9 28.0 17.8 13.3
Student Off Task Rate 6% or less 19.3 21.1 9.2 9.9
73
Now how black is the box?
Thank you for your attention