PowerPoint Presentation

A Framework for User Modeling in QuizMASterAthabasca University

Sima ShabaniOctober 2012What is QuizMASter?Online game show and a knowledge assessment toolGame-based learning environment can provide:ExcitementCompetitionPurpose of QuizMASter is for students to increase their knowledgeMulti users (learners) compete in one gameMulti Agent SystemComplexityUnpredictabilityTools, engaging, Two or more students simultaneously remotely log in to the system via a Web-based interface. Each student or learner is represented by one avatar in the virtual 3D world of QuizMASter. Students are able to view their own avatars as well as those of their opponents. Each game has a game show host who is also represented by an avatar visible to all contestants. The game show host asks each of the game questions from all the contestants. The students hear the voice of the host reading through each question; they also view the questions displayed on their screens. They individually and independently from one another answer each question by, for instance, selecting an answer from available choices in a multiple-choice format. Each correct answer would receive one mark.

2User Modeling In QuizMASterCreate and maintain models of all learners who participate in QuizMASter game showsPurpose of user modeling in QuizMASter is to create an adaptive environmentPurpose of creating an adaptive environment is to keep the students motivated during the game showStudent modeling an learner modeling are terms commonly used as an alternative to User modeling for the systems where the users to be models are students/learners. So they are correct terms to be used in our case of QM3Objectives of User Modeling in QMHelp pedagogical agents to:Select adequately challenging questions for each gameMatch the students with similar knowledge level for each gamePick informative and constructive feedback during the gameCreate personalized environment based on users characteristics and preferences

select and pose the questions whose levels of difficulty closely match the knowledge levels of game contestants in certain area of knowledge; meaning that the game questions for each game should be selected in the way that they are not too easy or too difficult for the contestants. The challenge here is to evaluate which one has more negative effects, the loss of interests by students who find the questions too easy or the ones who find the questions too difficult (discouraged). Easy bored no learning valueDifficult discouraged no learning value!

Challenges: Single user environment finding the question should not be that complicated, however finding the perfect question considering all contestants.

Tailored4Adaptive HypermediaUser modeling in QM to be based on adaptive hypermediaAdaptive Hypermedia in the form was introduced by Brusilovsky in 1996Definition: Adaptive hypermedia systems build a model of goals, preferences and knowledge of each individual user, and use this model throughout the interaction with the user, in order to adapt the structure and the content of the hypertext to the needs of that user (Brusilovsky, 1996)select and pose the questions whose levels of difficulty closely match the knowledge levels of game contestants in certain area of knowledge; meaning that the game questions for each game should be selected in the way that they are not too easy or too difficult for the contestants. The challenge here is to evaluate which one has more negative effects, the loss of interests by students who find the questions too easy or the ones who find the questions too difficult (discouraged). Easy bored no learning valueDifficult discouraged no learning value!

Challenges: Single user environment finding the question should not be that complicated, however finding the perfect question considering all contestants.

Tailored5Methods of User Modeling (Reminder)Classic methodsStereotype : Statistical demographics Highly Adaptive : Uniquely created for each user

InnovativeCombination of Stereotype and highly adaptive (Hybrid)Initializing the user models as they log in to the system for the first timeUpdating the user model as the users interact with the systemStereotypes: creating categories with certain characteristics each, classifying user models to one category personal attributes are not taken into account

Highly adaptive systems: there is an specific solution for each specific user and the systems behaviour can adapt to each single users need

A Framework for initialization of Student Models in Web-based Intelligent Tutoring Systems in 2004

6Methods of User Modeling in QMRecommending - HybridInitial student modeling Stereotyping (not discussed in the paper)General (a standard taxonomy) Athabasca university student specifically for QM (customized taxonomy)Adaptive user modeling (throughout the interaction with the system)Stereotypes: using patterns, categories to assign attributes to the users

Stereotype knowledge base

Athabasca university student specifically 7ExampleInitial Questioner

Name:Gender: Age:First Language:Level of Education:

Initial Questioner on the left

Stereotype pool on the right

8ExampleInitial Questioner

Name:Gender: Age:First Language:Level of Education:

John DoeMale43EnglishBachelor Degree

Initial Questioner on the left

Stereotype pool on the right

9ExampleInitial Questioner

Name:Gender: Age:First Language:Level of Education:

John DoeMale43EnglishBachelor Degree

Stereotype ABC

Gender: MaleAge: 40 to 45First Language: EnglishEdu.: Bachelor..English: 3.5 / 5Initial Questioner on the left

Stereotype pool on the right

Stereotype knowledge base

So even if we do not ask in the initial questionnaire as what is their level of English Knowledge as they are associated with the Stereotype ABC, we know that their English knowledge is approximately 3.5. This number will not change unless the student participate in a QM game of English Language and based on his/her score this attribute would change.

10Stereotyping for Athabasca University StudentsProgram EnrolledLevelYearMathPhysicsPhilosophyHistoryPolitical ScienceScienceUndergrad22.52.5111.5LiteratureUndergrad42143.53So, when student Jane Doe logs in for the first time, if she indicates that she is in Undergrad level year 4 and majoring in Literature, by stereotyping the pedagogical agents know that her knowledge of history is approximately 3.5, again if she competes in a game show in the field of history, based on her scores this number will be adjusted.

The benefits to stereotyping is that you dont have to get the users to fill out questionnaires with hundreds of questions or testing them in every field of study, by a few questions you can find their general stereotype and their student specific stereotype and have so much data about them. Although you might say that there is a problem with the accuracy of the numbers which is true but even in the self-assessment is not precise and accurate. We get to a more accurate results as they interact with the system more and more.11What is a User Model in QMInternal data structure with five group of fields containing information related to:Identification PersonalEducationalAffect and preferences QuizMASter specific12Identification InformationLogin ID (unique - assigned when user logs in first time)Password (purpose authentication for logging in)Student ID (future development of integrating QM with Moodle or other LMS)Personal InformationCollected when each student logs in to the system for the first timeNameAgePictureGenderFirst languageKnown languagesNationalityCountry of residence

Can be updated by the user UM system does not update it but uses the information for stereotyping 14Educational InformationCollected from the student or from the LMS DB in case of integrated systemsProgram student is enrolled inLevel (undergrad, grad)Courses taken - StatusFields of study (one or more fields to compete in QM) Level of knowledge in each field of study (an initial assessment questioner and/or stereotyping)Level of knowledge specifically for QM (initial value, updated by QM UM)Once this system is integrated with Moodle, the info can be retrieved from Moodles DB Science Math and physics

Initial Level of knowledge: Stereotyping or not? Initial value can be entered by the user? Novice, Beginner, Intermediate, Advanced, Expert

Level of knowledge for QM: initial value updated by QM UMS Students progress can be evaluated

Each field of knowledge that students are going to participate in, before the game they have to submit a simple initial assessment for the system to have an initial value for the knowledge, this will be updated by the systems pedagogical agents.15Affect and Preferences InformationPreferred style of learning (i.e. classroom, online, hybrid,)Preferred type of questions (in quizzes of QM)Level of enthusiasm for learningLevel of confidence Level of motivation

To have a unified answer a list can be provided to the users for example preferred type of quizzes (multiple choice, matching) it can be evaluated whether preferred style of learning would affect the degree of cognitive affect in students

This can be revisited by students, for example first time when they log in they associate a number to their level of enthusiasm for learning for example they are asked from 1 to 10 pick a number that represent your level of enthusiasm for learning from 0= I detest learning to 10: I adore learning and the rest in between. Then after each game, a small feedback can be taken from the participants how they enjoyed learning

Or for example, every time they log in, a pop up can appear saying You have marked your level of enthusiasm for learning as 4, has it changed since last time you used QM? If yes Has QM been an affecting factor in this change? Measuring tool for QM to see if it has been a positive factor or negative! (the same thing with confidence and motivation)

We can definitely use the stereotyping methods to give some initial value to some of these attributes, There are statistical analysis and studies out there mostly commonsense. For example, if you are a PhD holder your level of enthusiasm for learning should be technically high;)

16QuizMASter Specific InformationNumber of game shows attended so far (link to the show records)Score obtained in each game participated (average score in games of the same category/field)Average response time related to the questions of each game of the same categoryLets say Student A, a basic science major undergrad student, participated in 3 QM games. Two games in the field of math ad one in physics. Some important data that we need to keep records of are.

3 games, Scores obtained in game 1 math: 3/5game 2 math: 4/5Average math: 3.5/5

Game 3 physics: 2/5Average physics: 2/5

The average, factoring in the initial value for the knowledge level, represents the knowledge level of the student in the subject, the students knowledge level changes as they participate in more games.

Pedagogical agents collect data during the game shows and update this section of the user models.

17