FIT Learning Outcome WorkshopFIT Learning Outcome Workshop

Dr. Yeoh Eng ThiamDr. Yeoh Eng Thiam

11 & 13 August 201011 & 13 August 2010

AgendaAgenda

Briefing on OBE and targets to be achievedBriefing on OBE and targets to be achieved

Session 1: Revision of subject LOs and SLTsSession 1: Revision of subject LOs and SLTs Session 2: Mapping of Assessments to LOsSession 2: Mapping of Assessments to LOs

Session 3: Mapping of subject LOs to program POsSession 3: Mapping of subject LOs to program POs

Briefing on OBEBriefing on OBE

Curriculum, Teaching and Assessment Curriculum, Teaching and Assessment in Traditional Teachingin Traditional Teaching

CurriculumCurriculum TeachingTeaching Assessment Assessment

A list of topicsA list of topics Lecture, tutorialLecture, tutorial Exam, MCQExam, MCQto be coveredto be covered practicum practicum assignmentassignment

as defaults as defaultsas defaults as defaults

If students are to learn desired outcomes in a reasonably If students are to learn desired outcomes in a reasonably effective manner, then the teacher’s fundamental task is to effective manner, then the teacher’s fundamental task is to get students to engage in learning activities that are likely get students to engage in learning activities that are likely to result in their achieving those outcomes… It is helpful to to result in their achieving those outcomes… It is helpful to remember that what the student does is actually more remember that what the student does is actually more important in determining what is learned than what the important in determining what is learned than what the teacher does. teacher does.

(Shuell, 1986: 429)(Shuell, 1986: 429)

This is in fact a design for teaching:This is in fact a design for teaching:

1. 1. DDefine the efine the intended learning outcomesintended learning outcomes that refer not that refer not only to content to be learned, but what is to be done only to content to be learned, but what is to be done with that content and to what standards.with that content and to what standards.

2. 2. CCreate a learning environment that is likely to engage reate a learning environment that is likely to engage

the student in the student in learning activitieslearning activities that will bring about the that will bring about the intended outcomes. intended outcomes.

3. 3. Use Use assessment tasksassessment tasks that directly address the outcome that directly address the outcome and that enable you to judge if and how well students’ and that enable you to judge if and how well students’ performances meet the criteria.performances meet the criteria.

4.4. Transform these judgments into summative grades. Transform these judgments into summative grades.

Curriculum, Teaching and Assessment Curriculum, Teaching and Assessment in Outcomes-based teaching and learningin Outcomes-based teaching and learning

CurriculumCurriculum Intended learning outcomesIntended learning outcomes A list of topics to be covered BUT what are the students A list of topics to be covered BUT what are the students supposed to be able to do after learning those topics supposed to be able to do after learning those topics

that they couldn’t do before? What are that they couldn’t do before? What are the the intended intended learning outcomeslearning outcomes of teaching? of teaching?

TeachingTeachingWhat What learning activitieslearning activities should the students engage in should the students engage in order order for them to best achieve those outcomes? for them to best achieve those outcomes?

AssessmentAssessmentWhat assessment tasks (tests, assignments) would What assessment tasks (tests, assignments) would

really tell us really tell us how well how well they have achieved the intended they have achieved the intended learning learning outcomes?outcomes?

Teacher's and Student's Perspective on Assessment:Teacher's and Student's Perspective on Assessment:Outcomes-based teaching and learningOutcomes-based teaching and learning

TeacherTeacher: Intended Outcomes Teaching activities : Intended Outcomes Teaching activities AssessmentAssessment

StudentStudent: : Assessment Assessment Learning activities Learning activities Outcomes Outcomes

Implementing Outcomes-based Teaching and Implementing Outcomes-based Teaching and Learning using Constructive AlignmentLearning using Constructive Alignment

ILO: What the student

has to learn (apply

Psychology…)

Teaching:Engaging the student in the

verb in the ILO apply

Assessment:How well

the student has met the

ILO apply

Constructive AlignmentConstructive Alignment

Intended Learning Outcomes (ILOs)Intended Learning Outcomes (ILOs) expressed as verbs students have to enactexpressed as verbs students have to enact

The very best understanding that could beThe very best understanding that could be reasonably expectedreasonably expected: :

verbs such as hypothesise, apply to “far” verbs such as hypothesise, apply to “far” domains, generate, relate to principle, etc.domains, generate, relate to principle, etc.

Highly satisfactory understandingHighly satisfactory understanding: :

verbs such as explain, solve, understand verbs such as explain, solve, understand main ideas, analyze, compare, etc.main ideas, analyze, compare, etc.

Quite satisfactory learning, with under-Quite satisfactory learning, with under- standing at a declarative levelstanding at a declarative level: :

verbs such as elaborate, classify, cover verbs such as elaborate, classify, cover topics topics a a to to n,n,

Understanding at a low level Understanding at a low level : :

low level verbs, also inadequate but low level verbs, also inadequate but salvageable higher level attempts.salvageable higher level attempts.

Teaching / LearningTeaching / Learning ActivitiesActivities

Designed toDesigned to elicit desired elicit desired verbsverbs

May be:May be:

Large class activitiesLarge class activities

Small class activitiesSmall class activities

TeacherTeacher-managed-managed

PeerPeer-managed-managed

Self-Self-managedmanaged

as best suits contextas best suits context

Assessment TasksAssessment Tasks

FormatFormat such that such that the target verbs arethe target verbs areelicited and elicited and deployeddeployedin context.in context.

CriteriaCriteria clearly allow clearly allowjudgement as to thejudgement as to thequality of the quality of the student'sstudent'sperformanceperformance

Intended Learning Outcomes (ILOs) Intended Learning Outcomes (ILOs)

Higher Education InstitutesHigher Education InstitutesWhat are the HEI Learning Outcomes an ideal graduate What are the HEI Learning Outcomes an ideal graduate

of of the institution should achieve? the institution should achieve?

Programme Programme levellevelWhat are the intended learning outcomes for students What are the intended learning outcomes for students

enrolled in the degree programme?enrolled in the degree programme?

CourseCourse level levelWhat are the intended learning outcomes for students What are the intended learning outcomes for students

taking a particular course at a particular level within the taking a particular course at a particular level within the programme?programme?

Our TasksOur Tasks

To review subject LOs & SLTsTo align the subject assessments to the LOs To map the subject LOs to program LOs

Revision of Subject LOs and Revision of Subject LOs and SLTsSLTs

Course Intended Learning Outcomes (ILOs)Course Intended Learning Outcomes (ILOs)

•Statements of what students are expected to be able to Statements of what students are expected to be able to do as a result of engaging in the learning process do as a result of engaging in the learning process (studying a course).(studying a course).

•ILOs should reflect the level of the course.ILOs should reflect the level of the course.

•Expressed from the students' perspective (as different Expressed from the students' perspective (as different from from objectivesobjectives).).

•Expressed in the form of action verbs leading to Expressed in the form of action verbs leading to observable and assessable outcomes.observable and assessable outcomes.

•Related to criteria for assessing student performance.Related to criteria for assessing student performance.

The VerbsThe Verbs in the ILOsin the ILOs. Write ILOs by using appropriate . Write ILOs by using appropriate verbsverbs..

. Teaching is aimed at . Teaching is aimed at activatingactivating those verbs. those verbs.

. Students should be unable to complete the . Students should be unable to complete the assessment tasks unless they enact the same verbs assessment tasks unless they enact the same verbs that are in the ILOs (criterion-referenced). that are in the ILOs (criterion-referenced).

. Generic high level verbs include: . Generic high level verbs include: apply,apply, conceptualiseconceptualise, , reflectreflect, , create original insightscreate original insights, , solve solve unseen problemsunseen problems, , generate new alternatives, generate new alternatives, critically reviewcritically review. Such verbs might typically be used . Such verbs might typically be used to define an A grade in meeting the ILOs, depending to define an A grade in meeting the ILOs, depending on the subject. Low level verbs such as on the subject. Low level verbs such as describedescribe, , identifyidentify, would be more frequent in defining C and D., would be more frequent in defining C and D.

Distinguish the Distinguish the kindkind of knowledge you want of knowledge you want

DeclarativeDeclarative knowledge:knowledge:. Knowing . Knowing aboutabout things things. Knowledge we can declare to someone in . Knowledge we can declare to someone in writing or tellingwriting or telling e.g. ‘Distinguish between topic-based and e.g. ‘Distinguish between topic-based and outcomes-based teaching’ outcomes-based teaching’

FunctioningFunctioning knowledgeknowledge::. Knowledge we . Knowledge we put to workput to work in solving a in solving a physics problem, analysing a case study, physics problem, analysing a case study, designing a building, making an argument designing a building, making an argument e.g. ‘Write an ILO for a subject you are e.g. ‘Write an ILO for a subject you are

currently teaching’ currently teaching’

Alignment with teaching and assessment is Alignment with teaching and assessment is created by the created by the verbsverbs in the ILOs in the ILOs

For example: “Explain the historical evolution of nursing science”

Teaching is specifically aimed at activating the verb –for example the students do the explaining to each other, providing feedback from rubrics defining aspects of a good explanation (at end of this presentation). They don’t just listen to the teacher doing the explaining.

Students should be unable to complete the assessment tasks unless they enact the same verb that is in the ILO. For example, students could individually explain to the could individually explain to the class how they see the historical evolution of nursing. The class how they see the historical evolution of nursing. The teacher, perhaps using peer assessment too, assess each teacher, perhaps using peer assessment too, assess each explanation with the same rubrics.explanation with the same rubrics.

Procedures in designing Course ILOsProcedures in designing Course ILOs

1.1. Select the topics to be taught.Select the topics to be taught.

2.2. Decide what kind of knowledge is to be taught - Decide what kind of knowledge is to be taught - DeclarativeDeclarative or or functioning.functioning.

3.3. Decide the levels of understanding/performance the Decide the levels of understanding/performance the students are expected to achieve for the different students are expected to achieve for the different

topics.topics.

4.4. Consider if all the ILOs are of equal importance.Consider if all the ILOs are of equal importance.

5. Ensure a clear understanding and agreement of the 5. Ensure a clear understanding and agreement of the ILOs within the teaching team and other relevant ILOs within the teaching team and other relevant

parties, e.g. External Reviewer. parties, e.g. External Reviewer.

6. Communicate the ILOs to students.6. Communicate the ILOs to students.

ActionAction

1. Determine LO domain and level

2. Write the description using one of the relevant verbs (see reference list)

3. Check syllabus topics whether all LOs are covered accordingly

Designing Teaching/Learning Activities Designing Teaching/Learning Activities (TLAs) to Align with Intended Learning (TLAs) to Align with Intended Learning OutcomesOutcomes

Having designed Programme ILOs and the

Course ILOs, we now need to design

suitable Teaching/Learning Activities that

will facilitate students achieving the ILOs.

The best way to do this is to activate the verbs or learning activities embedded in the ILOs.

NB: there are many alternatives to lectures

and tutorials, even in large classes.

Typical ILOTypical ILO Possible TLAs Possible TLAs

Describe reading/lecture followed by presentationExplain tutorial, written essay, peer

assessmentIntegrate project, assignmentApply project, case study, work-based learningSolve problem PBL, case studyDesign, create project, creative writing Hypothesise experiment, projectReflect reflective diary

The point is not how you are going to teach but how and what you want your students to learn.

NOTE! Many of these TLAs can be assessments tasks as well. Then you have excellent alignment.

ActionAction

1. Determine the TLAs for each LO

2. Determine the duration for each TLA

3. Estimate the Self Learning Time (refer to SLT guidelines)

4. Calculate the subtotals and SLT

5. Calculate the Credit Hour (should match the subject credit hours)

Mapping of Assessments to Mapping of Assessments to LOsLOs

Assessment Tasks (ATs)Assessment Tasks (ATs)

.. provide students the opportunity to demonstrate provide students the opportunity to demonstrate whether or not they have achieved the ILOs and what whether or not they have achieved the ILOs and what

level their performance is in those ILOs. level their performance is in those ILOs.

.. should be appropriately designed or selected to should be appropriately designed or selected to address the address the ILOs that we want to assess.ILOs that we want to assess.

.. different assessment methods (tasks) address different different assessment methods (tasks) address different ILOs. There should therefore be several kinds of task. ILOs. There should therefore be several kinds of task.

.. provide the evidence allowing teachers to make a provide the evidence allowing teachers to make a judgment about the level of a student’s performance judgment about the level of a student’s performance against the ILOs and to award a final grade. against the ILOs and to award a final grade.

Common ILOs Common ILOs Possible Assessment TasksPossible Assessment Tasks

DescribeDescribe essay question, exam, oral essay question, exam, oral presentation (peer assessment)presentation (peer assessment)

ExplainExplain assignment, essay question assignment, essay question exam, oral, letter-to-a-friendexam, oral, letter-to-a-friend

IntegrateIntegrate project, assignmentproject, assignmentAnalyseAnalyse case study, assignmentcase study, assignmentApplyApply project, case study, experimentproject, case study, experimentSolve problemSolve problem case study, project, experimentcase study, project, experimentDesign, createDesign, create project, experimentproject, experimentReflectReflect reflective diary, portfolio,reflective diary, portfolio,

self-assessmentself-assessmentCommunicateCommunicate a range of oral, writing or a range of oral, writing or

listening tasks, e.g. presentation, listening tasks, e.g. presentation, debate, role play, reporting, debate, role play, reporting,

assignment,assignment, precis, paraphasing, precis, paraphasing, answering questions etc.answering questions etc.

ActionAction

1. Define Learning Activities/Assessment Methods

2. Map Assessment to LOs

3. Determine the percentage of each assessment (total should be 100%)

4. Calculate the percentage contribution of each assessment to LOs

Mapping of LOs to POsMapping of LOs to POs

Program OutcomesProgram Outcomes

Every program has a set of POs to define the student who completes the program

The POs are achieved through the courses in the program

(Note: in next diagrams, LOD = PO)

ActionAction

Map each LO to the relevant POs

ReferencesReferencesBiggs, J. B. and Tang, C. AKEPT Workshop: “Outcome-based

Teaching and Learning: Enhancing Learning Through Constructive Alignment”, Feb 2010.

Biggs, J. B. and Tang, C. (2007, 3rd edition) Teaching for Quality Learning at University, Maidenhead, UK: Open University Press/McGraw Hill, 2007.

Fernandez-Chung, R.M. MQA Training: “Planning to Achieve Learning Outcomes”, Apr 2010.

Wong, H.Y. MMU Workshop: “OBE Measurement”, Jul 2010.