Service Oriented Service Oriented ArchitectureArchitecturein eTesting Systemsin eTesting Systems

Institute of InformaticsInstitute of Informatics - - FNSFNSUniversityUniversity “Ss“Ss. . Ciryl and Methodious”Ciryl and Methodious” – – Skopje, MacedoniaSkopje, Macedonia

66thth Workshop Software Education and Reverse Engineering Workshop Software Education and Reverse Engineering

Goce Armenski, M.ScGoce Armenski, M.Sc(armenski@ii.edu.mk)(armenski@ii.edu.mk)

1.1.

CONTENTCONTENT

ContentContent

- - CCONCLUSIONONCLUSION

- - CCOMMON FRAMEWORKOMMON FRAMEWORK

- - ЕЕТТЕСТЕСТ – – SSYSTEM FOR ELECTRONIC TESTINGYSTEM FOR ELECTRONIC TESTING

- - SSERVICE ORIENTED ARCHITECTUREERVICE ORIENTED ARCHITECTURE

- - IINTRODUCTIONNTRODUCTION

- - ААRCHITECTURERCHITECTURE, , CONCEPTS AND FUNCTIONALITY OF CONCEPTS AND FUNCTIONALITY OF ЕЕТТESTEST

- - AAPPLICATION OF EPPLICATION OF ETTESTEST

- - RRESULTSESULTS

- - SOA SOA IN EIN EAASSESSMENTSSESSMENT

2.2. еТеТestest – – concepts and functionalityconcepts and functionality

еТеТest Technologyest Technology

- Web based applicationWeb based application- - Active Server Pages (ASP)Active Server Pages (ASP)- JavaScript- JavaScript- SQL Server 2000- SQL Server 2000- NT Server - NT Server andand Win 2000 Win 2000 compatiblecompatible- IIS 4.0 - IIS 4.0 or neweror newer

Web based sollutionsWeb based sollutions VS desktop based sollutionsVS desktop based sollutions

Web Browser(Netscape 4.x or Internet Explorer 4.x, and above)

Win 2000 Server

Email(SMTP)

IIS 4.0 + ASPSQL ServerAccess

JScr

ipt

3.3. еТеТestest – – concepts and functionalityconcepts and functionality

- learning objectslearning objects- - three structurethree structure

Course organizationCourse organization

Fig. 3 Course organization

LECTURE

PARTA PART PARTB C

SET SET SET SETA1 A2 A3 A4

4.4. еТеТestest – – concepts and functionalityconcepts and functionality

• Multichoice questionsMultichoice questions ( (choose one of many, choose many of many, yes/no choose one of many, choose many of many, yes/no answersanswers););

• Short entry answerShort entry answer ( (text or numericaltext or numerical););

• Essay answerEssay answer..

Types of questionsTypes of questions

• Questions can have pictures or graphs in the text or offered answersQuestions can have pictures or graphs in the text or offered answers

5.5. еТеТestest – – concepts and functionalityconcepts and functionality

- dynamic linear testsdynamic linear tests ( (fixed number of questionsfixed number of questions))

Marking and reportingMarking and reporting

Test creation algorithmTest creation algorithm

- results are shown at the end of the testresults are shown at the end of the test- - negative markingnegative marking

Fig. 4 Algorithms for test creation

Linear Dynamiclinear

Testlets Masterymodels

Adaptive

Level of adaptation of the test

6.6. еТеТestest – – concepts and functionalityconcepts and functionality

Statistical data analysesStatistical data analyses

• Identification of content which is not well presentedIdentification of content which is not well presented;;

• Personalized feedback to studentsPersonalized feedback to students;;

• Identification of week questions which need to be revised Identification of week questions which need to be revised before used againbefore used again;;

• Identifying the individual weaknesses of studentsIdentifying the individual weaknesses of students..

7.7.

44. . APPLICATION OFAPPLICATION OF еТ еТESTEST

Application of eTestApplication of eTest

- - Integration in the process of learningIntegration in the process of learning- - Controlled learningControlled learning

проучиА1

тестирајА1

проучиА2

тестирајА2

проучиА3

тестирајА3

проучиА4

тестирајА4

проучиА

( ,А А А А1 2 3 4, ,

)

тестирајА

( , )А А А А1 2, 3, 4

Fig. 5 Way of passing the learning objects

Successful strategySuccessful strategy????- all questionsall questions- NN questions in a rowquestions in a row- NN right questionsright questions- 3 3 right questions in a rowright questions in a row

Statistical analyses of the user activitiesStatistical analyses of the user activities

8.8.

55. . RESULTSRESULTS

ResultsResults

The use of technology in education is very dependent of The use of technology in education is very dependent of the organizationthe organization - logistics - logistics - social changes - social changes

Synchronization with other systemsSynchronization with other systemsTechnical infrastructureTechnical infrastructure

Practical ImplementationPractical Implementation

• Institute of InformaticsInstitute of Informatics , , FNSFNS (2001) (2001)

• А.А.DD. . MobimakMobimak (2002) (2002)

• UNDPUNDP (2003-2004) (2003-2004)

9.9. ResultsResults

Institute of InformaticsInstitute of Informatics, , FNSFNS ( (januaryjanuary 200 20011))

- 26 courses- 26 courses

- - 1239112391 questionsquestions

- 589 scheduled assessments- 589 scheduled assessments

- 9861 generated tests- 9861 generated tests

Data gatheringData gathering

Does eTesting provides more effective and more objective Does eTesting provides more effective and more objective assessment compared to the traditional forms, and does it help the assessment compared to the traditional forms, and does it help the learning processlearning process

10.10. ResultsResults

Installed in 4 Faculties from 3 UniversitiesInstalled in 4 Faculties from 3 Universities

- not interoperable- not interoperable

- does not allow searching or using common question bankdoes not allow searching or using common question bank

- does not allow cross-institutional cooperationdoes not allow cross-institutional cooperation

- not fully standards compliantnot fully standards compliant

Problems Facing withProblems Facing with

Possible SolutionPossible Solution

11.11. ResultsResults

12.12.

6. SERVICE ORIENTED ARCHITECTURE6. SERVICE ORIENTED ARCHITECTURE

Service Oriented ArchitectureService Oriented Architecture

IT Architecture RequirementsIT Architecture Requirements

• SimplicitySimplicity

• Software that reflects the Software that reflects the need of businessneed of business

• Agility and flexibilityAgility and flexibility

• ReusabilityReusability

• Technology independentTechnology independent

• Promote integrationPromote integration

Built to Change

Evolution of the WebEvolution of the Web

Generation 1Static HTML

HTMLHTML

Generation 2Web Applications

HTMLHTML

HTML, XMLHTML, XML

HTML, XMLHTML, XML

Generation 3Web Services

13.13. Service Oriented ArchitectureService Oriented Architecture

14.14. Service Oriented ArchitectureService Oriented Architecture

An approach for buildingAn approach for building distributeddistributedcomputing systemscomputing systems based on based on

encapsulatingencapsulating business functionsbusiness functions asas servicesservices that can be easily accessed that can be easily accessed

in ain a loosely coupledloosely coupled fashion.fashion.

SERVICE ORIENTED ARCHITECTURESERVICE ORIENTED ARCHITECTURE

SOA: It’s Not a ThingSOA: It’s Not a Thing

• ProductProduct

• A specific technologyA specific technology

• An applicationAn application

• A specific standardA specific standard

• A specific set of rulesA specific set of rules

It’s not a:It’s not a:

An Approach for Building Agile and Flexible An Approach for Building Agile and Flexible Business ApplicationsBusiness Applications

15.15. Service Oriented ArchitectureService Oriented Architecture

7. SOA in eLearning7. SOA in eLearning

16.16. SOA in eLearningSOA in eLearning

• JISC Technical Framework to support e-LearningJISC Technical Framework to support e-Learning ( (ELFELF))..

• IMS Abstract Framework (IAF)IMS Abstract Framework (IAF)

• LeAPP Learning Architecture ProjectLeAPP Learning Architecture Project

• Carnegie Mellon’s Learning Services ArchitectureCarnegie Mellon’s Learning Services Architecture

e-Learninge-Learning ( (ELFELF))

17.17. SOA in eLearningSOA in eLearning

8. SOA in eAssessment8. SOA in eAssessment

18.18. SOA in eAssessmentSOA in eAssessment

• FREMA.FREMA.

• TENCompetence Assessment ModelTENCompetence Assessment Model

FREMA Core ServicesFREMA Core Services

• Assessment ServicesAssessment Services– AssignAssign– Author itemAuthor item– Author assessmentAuthor assessment– Validate Validate

assessmentassessment– Take assessmentTake assessment– Mark assessmentMark assessment– Moderate Moderate

assessmentassessment– Grade AssessmentGrade Assessment– View FeedbackView Feedback

• Support ServicesSupport Services– ScheduleSchedule– NotifyNotify– TrackTrack– Authorise and Authorise and

AuthenticateAuthenticate

Assessment Domain Definition

Use CasesUse CasesUse Cases

Service Profiles

Gap Analysis

Reference Impl’

Common Usage Patterns

19.19. SOA in eAssessmentSOA in eAssessment

TENCompetence - Phases in the assessment TENCompetence - Phases in the assessment processprocess

Assessment DesignAssessment Design

ItemItemConstructionConstruction

AssessmentAssessmentConstructionConstruction

AssessmentAssessmentDeliveryDelivery

ResponsResponsEvaluationEvaluation

DecisionDecisionMakingMaking

20.20. SOA in eAssessmentSOA in eAssessment

eTest Core ServiceseTest Core Services

• Assessment ServicesAssessment Services

– Item ConstructionItem Construction– Test ConstructionTest Construction– Test DeliveryTest Delivery– Results CollectingResults Collecting– MarkingMarking– Decision MakingDecision Making– Statistical AnalysisStatistical Analysis

• Support ServicesSupport Services

– ScheduleSchedule– Notify and AnnounceNotify and Announce– TrackTrack– AuthenticationAuthentication– User ManagementUser Management

21.21. SOA in eAssessmentSOA in eAssessment

22.22.

SOA SOA Projects in eAssessmentProjects in eAssessment

SOA in eAssessmentSOA in eAssessment

– Research ProjectsResearch Projects• TOIA – Technologies for Interoperable TOIA – Technologies for Interoperable

assessmentassessment

• ASSIS – Assessment SequencingASSIS – Assessment Sequencing

• APIS – assessment Provision through APIS – assessment Provision through InteroperabilityInteroperability

• … … and many othersand many others

– Standards BodiesStandards Bodies• In Assessment – IMS, OSIDs … In Assessment – IMS, OSIDs …

• In Web Services - SOAP, WSDL, WSRF, W3C …In Web Services - SOAP, WSDL, WSRF, W3C …

23.23.

66. . CONCLUSION and FUTURE FORKCONCLUSION and FUTURE FORK

ConclusionConclusion

Service Oriented ArchitectureService Oriented Architecture• Advantages over monolithic applications for TestingAdvantages over monolithic applications for Testing

Review of existing SOA models for eAssessmentReview of existing SOA models for eAssessment

• Design of Common Framework by service identificationDesign of Common Framework by service identification

• Practical realizationPractical realization

Extend the ASSIS project with different test delivery modelsExtend the ASSIS project with different test delivery models

• Dependent on contextDependent on context

• Game based learningGame based learning

Thank you for Thank you for your time!your time!