eTest Solution: Software for Integrating Large- and

Small-scale Assessment

Srdjan Verbić1, Srdjan Božović2, Saša Veličković2

1 Institute for Education Quality and Evaluation, Belgrade2 MFC Mikrokomerc, Belgrade

18th July 2012 @ Lisbon

Context of e-assessment project

still undeveloped Internet infrastructure just adequate computer classrooms MS Windows only available free software lack of experience curious students unmotivated and

untrained teachers

Typical small- and large-scale tests

Small-scale Large-scale

Sample size classroom hundreds of schools

Example final exam standards setting, PISA, TIMSS

Examinees all students representative sample

Purpose mostly summative mostly formative

Measure how much student knows

what competences students have

Risk-level high-stakes low-stakes

Trial-testing

Trial testing (or pilot-testing)

the same conditions as for final testmore-less the same questionssmaller sample (usually volunteers)(very) low-stakeslooking for items’ characteristics

meeting point of assessment

professionals and teachers

Prehistory of the project

Moodle testing machine, 2007Moodle large-scale on-line test, 2009

Large-scale e-assessment 2007(the evil plan)

put Ttest instead of Quiz adjust Moodle distribution distribute Moodle installation to schools encourage teachers to use it ask them to download test :) collect data

Never-ending series of problems

Moodle is not so good solution for on-line tests, after all.

memory leaking, again unreliable Internet connections slow update of the awkward database …

New system 2011

Microsoft technology

XML based interoperability (like IMS QTI)

Separate system components

On- and off-line testing

Efficient use of multimedia

Digital Schools

2808 schools (95% of all schools in SRB)30,000 working places

2 Multipoint Servers and16 workstations per clssrm

Unfortunately, broadband connection is not included!

htt

p:/

/dig

italn

ask

ola

.rs/

eTest Solution

eTest PlayereTest School PortaleTest Management PortaleTest Question Editor

•SQL Server as database•IIS as Web server•ASP.NET as Web application

platform•Silverlight as a browser plug-

in that for hosting eTest Player

htt

p:/

/mfc

.rs/

Example of unexpected results

Questionnaires’ feedback

Students prefer e-testing mode……and don’t have problem to

communicate with computers.“collective behavior”Teachers agree that <30% of colleagues

would be able to conduct e-testing.Main obstacle: untrained teachersMain challenge: to prepare enough high-

quality tests ready for download

Further steps

Teachers’ training

Installing software in all digital classrooms

More training

Providing high-quality tests