Technology Enhanced LearningTest and Quizzes in Sulis

Facilitator: Angelica Risquez

Index

Benefits of T&Q T&Q key features T&Q homepage Question Pools New assessment: Quick create New assessment: Slow create Changing and previewing assessments Publishing assessments Working with scores and answers Retracting a published assessment Exporting/importing assessments Assessment types

Benefits of online formative T&Q

Learners develop understanding progressively and have opportunities for feedback and reinforcement

Allows for scalable formative testing and feedback provision that would not be possible without automatisation

Available anytime, anywhere You can delegate on students to create questions If well used, may reduce teachers’ workload and increase

students’ responsibility for their own learning It may increase participation in face-to-face sessions

Useful case: http://www.elearning.ac.uk/effprac/html/casestudy5.htm

Test & Quizzes in Sulis: Key FeaturesAutomatic grading: True/false, matching, multiple choice, fill-in-the-

blank, numeric response, survey (grades automatically transferred to the Gradebook tool)

No automatic grading: short answer/essay and file uploadQuestion Pools to re-use questions and share them Import assessments from other sites you own and import previously

created tests/quizzes/surveys (QTI -compliant files) Create questions with online forms (standard approach) or single text

box (quick create approach)Customisation: location for taking a test, IP addresses, availability of

test, time limits, number of submissions allowed, and late submissionsAttachments (by uploading local files, specifying URLs, and selecting

files from Resources) to any question.Templates (or “assessment types”) to speed customisation

T&Q Homepage (instructor view)1. Core Assessments (unpublished) and Published ass’s.2. You can return to this screen with the reset arrow!3. New assessment options

1. Quick Create2. Create (recommended)3. Import (xml file)

Question Pools: create In the T&Q tool, click the Question Pools button. In the bar near the top

of the screen, click Add New Pool You can also import an assessment as a question pool, after exporting

an assessment from another site:In the T&Q home page (instructor view) click Question Pools. In the bar above the list of pools, click Import and follow instructions to import your assessment

You can also create subpools, move and delete pools To add questions to a pool or subpool, click the “Add” link (right hand

side of the screen, opposite to “Questions”), and follow steps To Copy a question from a Question Pool to an assessment, you use the

pulldown list on the Questions screen (see slide 6) to select the type of question to add. Choose the option “Copy from Question Pool”

New Assessment: “Quick” CreateOnly for multiple choice (single or multiple answer), true/false, survey and fill-in-the-blank questions. 1.Name the assessment and click “Quick Create” (don’t choose assessment type for now). Sulis will display the Questions screen2.Click the links in for each type of question to read specific instruction and examples3.Review each question on the Validate Assessment/Question Pool screen4.Click “create assessment”: Sulis displays the T&Q home page and the new assessment listed under Core Assessment (you can create a Question Pool instead)

When you finish with the process, your new quiz will appear under “Pending assessments”. Note: the quiz is not yet available to students!

New Assessment: “Slow” CreateTo be used for other type of questions, to create parts (needed for random assessments), and to reuse questions from Question Pools

Name the assessment and click “Create” (don’t choose assessment type for now). From menu “Add question”, add questions one by one or bring from Question Pool

When you finish with the process, your new quiz will appear under “Pending assessments”. Note: the quiz is not yet available to students!

Create questions You can either create new questions in your assessment or add questions from a

Question Pool. You may add question to any unpublished assessment. Under the appropriate part, select the type of question you want to add with the “Add

Question” pull down menu Answer point value: If questions have two correct answers, the points for each correct

question accumulate (e.g. 10+10=20). For ungraded questions enter 0 (zero). Incorrect answers are worth negative points.

Using the Assign to Part pulldown list, select the part (that is, the section) of the assessment to which you want to assign this question

To assign a question to a question pool, select the name of the desired pool from the Assign to question pool pulldown list.

To provide question-level feedback enter text in the boxes under “Correct/Incorrect Answer Feedback.” (voluntary)

You can require participants to provide a reasons for choosing an answer under Required Rationale

New Assessment: “Slow” Create (2)

Create questions (continued)Multiple choice questions To indicate the correct response, click the Correct Answer radio button for that

response (multiple correct answers allowed) To display the possible answers in random order, und Randomize AnswersSurvey choice questions You do not have the option of randomizing answers or requiring a rationale.Short answer/essay questions Answer point value: enter the point value you want to assign to this question. For

an ungraded question, enter 0 (zero). (MANUALLY GRADED)Fill the blank questions Place curly braces { } around words requiring a blank For example: {gravity}. If more than one response is correct for a given blank, or to allow alternate

spellings, use commas to separate the alternate responses. For example: Darth Vader actually was Luke Skywalker's {father, dad, poppa, pater, pa, daddy}.

For questions with more than one blank, each blank must contain a unique response (mutually exclusive). For example: The colors of the Irish flag are {orange, white, green}, {orange, white, blue}, and {red, white, blue}. To answer correctly, the participant must respond with some combination of orange, white, and green.

New Assessment: “Slow” Create (3)

Create questions (continued)Numeric Questions Place curly braces { } around numbers that require a blank (for example: 3*3=

{9} y 2+2= {4} requires a blank before the digits 9 and 4). If a range of values is acceptable, insert a pipe between the highest and lowest

acceptable values (for example: 5|8 accepts answers of 5, 6, 7, and 8).Matching Questions In the Choice box, enter a choice. In the Match box, enter the correct match

for the choice you specified. Click the Save Pairing button. Sulis saves the pair you have created and

displays it under the Add/Edit Pairing and Optional Feedback header File Upload Questions In the Answer Point Value box, enter the point value you want to assign to

this question. For an ungraded question, enter 0 (zero). (MANUALLY GRADED)

You may change questions in any unpublished assessment. Display the Questions screen for the assessment, find the question you want to change and click the Edit option for it.

New Assessment: “Slow” Create (4)

To create a random quiz from a question pool, you need to edit the settings of a part of the test. The part should be empty of questions, if it is now, you need to create a new part (click “Add Part”). You can use the Default part as long as no questions have been added yet: click “Edit” for Default Part and under Type, choose “Random draw from question pool” (choose pool).

New Assessment: “Slow” Create (5)

Changing and Previewing Assessments You may change the settings, parts, and questions for an assessment at any time BEFORE it is published (unpublished assessments are listed under Core Assessments on the T&Q home page) After an assessment has been published, you may change only the following attributes:

Delivery, Due, and Retract dates Gradebook options Feedback Delivery options Feedback Components options

To preview an assessment before publishing it, display the Questions screen for the assessment and click the Preview Assessment link By default, only students or site members with the appropriate permissions may submit an assessment. Instructors and site organizers may preview an assessment but may not submit an assessment.

Publishing Assessments When you publish an assessment, you make it available to your site's

participants. After the assessment has been published, its contents CANNOT be changed.

1. In the Core Assessments section, click the Settings link under the title of the assessment you want to publish.

2. Work through the sections by opening them and choosing the appropriate options for this assessment.

To open all sections, click the Open link under the title of the screen. To close all sections, click the Close link under the title of the screen. To open a section, click the triangle icon (►) for it. To close a section, click its triangle icon (▼) again.

3. Make sure all settings are configured as you want them. Be sure to click Save Settings and Publish. If you click Save Settings, the quiz will NOT be published.

4. Sulis displays a confirmation screen. Click Publish.

Publishing Assessments (2)

Working with Scores and AnswersIn the Published Assessments section, find the assessment you want and click the Scores link under its name. Sulis displays the Total Scores screen

You entered this info

Sulis generated this total

To adjust a student's total score and add an overall comment, enter an adjustment to the score in the Adj box. To add a comment, enter it in the Comment box. After entering changes for all appropriate participants, click Update. Sulis calculates and displays the final scores.

Working with Scores and Answers (2) To see the submission status of an assessment for all

participants, click the Submission Status button

If the Date entry is blank for a student, s/he has not submitted the assessment. You may contact any student who is expected to submit an assessment by sending him or her an email (Click

the Email link under the student’s name) To allow one or more students to retake an assessment, click the Allow retake? link under the student’s

name

Working with Scores and Answers (3) To see individual questions and all responses to it to quickly

compare all participants' answers to any question, click the Questions option

The Question section displays the question, the number of points allotted to it, all possible answers, and the correct answer(s) The Responses section lists all participants by user name, date the assessment was submitted, score on the question, student's

response, and a box for comments. You may view responses, view the highest scoring submission for each student or all submissions, and view each response. You can also download all files uploaded by students in response to file upload questions

You can also adjust a student's score and add a comment for one or more individual questions

Working with Scores and Answers (4) To see statistics for the assessment as a whole and for each

question and its responses, click the Statistics option

The Total Score section shows the total number of submissions and the total score possible on the assessment, along with statistical information such as the mean, median, mode, and range of the scores for the assessment as a whole.

Below, Sulis displays statistics for each question and its possible answers (with a check mark for the correct answer ), a histogram showing the number of students who selected each response, the total number of responses to the question and the percentage of participants who gave the correct response.

You can also see an item analysis for each question

Working with Scores and Answers (5)

… and also download all questions and scores into Excel

Retracting a Published Assessment To remove a test from students’ view, you need to retract it. After you have retracted a published assessment, you can change the settings, parts, and questions in it, as

needed, and then republish the assessment. If you change a retracted assessment and republish it, the newly published assessment will have a different URL

1.Find the assessment in the Published Assessments section 2.Click the title of the assessment and click Edit to retract

Exporting Assessments You may want to export an assessment to upload in another

course or share with a colleague1. In the Pending Assessments section and click the Export link under the title of the quiz, then select QTI v 1.2 and

click Export

2. Your browser displays a window with downloading instructions (click Download; after the browser displays the XML file, click File/Save Page As, enter a name for the new file, and save it in your pc)

Importing AssessmentsYou can share quizzes among instructors with the export/import features1. Lecturer A follows instructions to export quiz (previous slide), and sends the file to Lecturer B2. Lecturer B goes to T&Q home page (instructor view). In the New Assessment section, clicks the Import Assessment button and follows instructions.

Note: you can use this to reuse quizzes from another course, but using Question Pools is a preferred solution

Assessment TypesAssessment Types are collections of settings (no questions) that are useful

if you are going to have publish several of the same type of test

Predefined types (choose existing assessment type list). You can edit them if you want. Please check their settings carefully before choosing one! (e.g. grading options?)

You may create your own:1. Click Assessment Types button2. Type a name for new assessment type, click create. 3. Choose appropriate options in each section (click ►to open

each section). Be careful with “can edit default value”

Questions?

Angélica RisquezTechnology Enhanced Learning Advisor

Centre for Teaching and LearningMillstream CourtyardUniversity of Limerick

Limerick, Ireland[T] +353-61-202 580[F] +353-61-338 044

[E] angelica.risquez@ul.ie[W] www.ul.ie/ctl