learning and teaching week 01 september 2015 developing quality learning outcomes dr hana morrissey...
TRANSCRIPT
Slide 1
Developing Quality Learning Outcomes
Dr Hana Morrissey
Dr Brian Phillips
Dr Bopelo Boitshwarelo
Amanda McCubbin
Charmain Enthaler
Learning and Teaching Week 01 September 2015
1
In this session we will cover the process of developing learning outcomes from the point of naming the course to designing assessment items in each of its units.
Overview
2
Have you ever thought why learning outcomes are one of the most discussed item during the course accreditations process?
3
What we need to consider when developing learning outcomes?
Standards/benchmarking
External bodies, Discipline or Profession competency standards
University requirement
Graduate needs or Employability attributes
Knowledge
Foundation knowledge
Knowledge gap
Advanced knowledge
Speciality
4
How we can consider all of that to come up with only 4-8 learning outcomes, it is a big challenge.
5
Does the course name mean the same of what the course will produce a key ring designer?
What is the knowledge or science behind key ring designing that the course must cover?
What are the skills or competencies the key ring designer must be able to do?
Hypothetical Example: Bachelor of Key Ring Designing
6
What this key ring designer students must know in theory in the class/online and what they need to learn hands-on at the workplace?
Who this key ring designer will work with, for or communicate with and in which setting or environment?
How this course can make the graduate key ring designer more employable and better than all other key ring designers?
Hypothetical Example: Bachelor of Key Ring Designing
7
Brian will talk about the value of articulating the main principles and quality of a course
Understanding the course you are designing from both the educator and the student views
8
Bopelo will demonstrate how we can use those course main principles to develop learning outcomes
Using the course main principles or quality to design learning outcomes
9
Amanda then will show case of how we can use a learning outcome to design assessment tasks
Using the course learning outcomes to design assessments
10
Charmain will lead the question and answer session and summarise the main points we covered today.
Summary
11
So let us get started
12
Achieving learning outcomes through understanding the course designBrian Phillips
using the 240cp bachelor degree as the reference point for this session
Learning and Teaching Week 01 September 2015
I will generally talk in terms of the Bachelor Degree as the example or reference point.
Talk as if everyone is a course designer including me. This is how I make sense of courses and how a proposal might be defended to accreditation authorities; including our own committees.
13
What do we want our course graduates to demonstrate as course outcomes?
How to get them there? (i.e. course design)
The vision thing
I want to start at the beginning. Concretely, why bother about course design? because we have a vision of the graduate we want our course to produce. And the course needs to be able to do this.
How are we getting our students from A to B? Note I am asking about the pathway at this point rather than a list of topics bear with me.
To get students from A to B, we need to design backwards for B.
14
Answer: a programme of learning
Designing a programme includes considering:
Structure
Components
Sequencing
What is a Course?
Does it make sense educationally?
Does it make sense to the discipline/profession?
But first, what is a course?
We are not free in an entirely unlimited sense as to the structure, the components (units), and sequencing. In other words we cant randomly gather 24 units and throw them into a bowl, toss lightly, and see how they fall out! We have to comply with Common Course Rules for instance.
But the biggest issue is ourselves as a group of academics running the course.
The biggest factors in my view that determine what might be in our course is (1) ownership by the discipline and (2) an overarching educational logic as to how students experience their learning pathway through whatever these 24 units are. Ownership by the discipline ultimately means the course has to acceptable to the discipline internally and externally, and internationally. Yes, Im talking about peer review and benchmarking.
An overarching educational logic or strategy means that we are explicit in the educational processes built into the course that describes how Susan and Robert will be guided to get from entry level to graduation.
So, the job of each unit in the overall structure is explained by both its role in the discipline AND in our educational strategy.
So far, does this make sense to the audience? If so, I want to next consider structure, components and sequencing??
15
How are we to get our students from entry-level capability to ultimately demonstrate what we intend them to achieve at the final level?
Designing backwards
To go from A to B, we need to start at the end and work our way backwards. Our students travel from A to B, but to design, we need to go from the end point backwards.
Once we do the vision thing, we end up having approximately six clear (well-debated!) statements that will now guide everything about the course (easy-peasy!) Intended Learning Outcomes.
My own preference is to think of the traditional 240cp bachelor structure as providing a student learning pathway through scaffolding units into final level units where they finally demonstrate their achievements at bachelor level.
16
Scaffolding
Transitional unitsIntermediate/advancingFinal level unitsStudent learning experience
Broadly speaking the three years of a standard 240 cp bachelor degree can be scaffolded like this diagram.
Note the final level is at the bottom. By keeping the final level outcomes in mind we work backwards to build the foundations and scaffolding up from there.
In my mind this has strong echoes of Blooms taxonomy perhaps inverted:
Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
17
Start from the end point the learning outcomes.
Then build the scaffolding for the student to realistically achieve them.
Start designing backwards!
So.. When reviewing or designing a course we probably need to do it backwards.
There is a bit more detail, because scaffolding involves putting 24 units together in a structure and sequence that the student can successfully navigate.
18
The structure is 24 units fitted together into an integrated whole (i.e. coherence) that makes sense
To the discipline/profession,
From an educational perspective, and
From the perspective of the student/learner experience.
Structure and sequence
Instead of merely a list of 24 topics that can be undertaken at any time in any order with equal sense of achievability, what is it that would make disciplinary peers see that it is a coherent educational experience for the student?
Ive tried to depict this visually in the next slide.
19
Sequencing
Transitional unitsIntermediate/advancingFinal level unitsStudent learning experience
Disciplinary / professional
Educational
Broadly speaking the three years of a standard 240 cp bachelor degree can be scaffolded like this diagram.
Hopefully this diagram depicts the logic to the scaffolding structure and sequence. The vertical lines/pillars represent the argument or academic defense of the structure and sequencing. So, we have twin overarching reasons for having certain units early in the degree, in the middle, or at the end.
This picture suggests that we cant lose sight of (1) the educational logic, (2) the needs of the discipline, and (3) how the student is expected to navigate their learning experience.
In my mind this echoes Blooms taxonomy:
Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
20
Presentation Title | 00 Month 2010 | Slide #
21
Fundamentally these are our units.
Levels of units
Map to each other in order to scaffold the learning experience
Components
Units are defined in the CDU Units and Courses Policy. Our main interest here is that units have levels that indicate their role in scaffolded learning and generally align with the AQF level of qualifications.
So let me revisit my final diagram to put this last detail onto it.
22
Components (units)
100-level (Diploma) - transitional200-level (Adv Diploma) intermediate scaffolding300-level (Bachelor) final level outcomesStudent learning experience
Disciplinary / professional
Educational
Now we have the outline of our bachelor degree.
23
Australian Qualifications Framework Council. (2013). Australian qualifications framework (2nd ed.). South Australia: Author.
Biggs, J., & Tang, C. (2011). Teaching for quality learning at university: what the student does (4th ed.). Maidenhead, UK: Open University Press.
Charles Darwin University. (2013). Course common rules bachelor degree.
Charles Darwin University. (2014). Units and courses policy.
Donnelly, R., O'Neill, G., & Fitzmaurice, M. (2013). Supporting programme teams to develop sequencing in higher education curricula. International Journal for Academic Development. doi: 10.1080/1360144X.2013.867266
Transition Pedagogy website at http://transitionpedagogy.com
My Sources
24
Developing Learning Outcomes
Learning and Teaching Week 01 September 2015
In this part of the session we will explore how to develop learning outcomes
We will start by discussing:
What LOs are
Why we need LOS
Characteristics of good LOs
And to help us conceptualise our own LOs we will look at components of a learning outcome and the structure of a learning
And then we will have a go at drafting LOs
The session is going to be a touch and go and the idea is really to introduce to some ideas around developing LOs which I hope will be helpful and can be explored beyond this workshop.
I want to also acknowledge that most of you may have experience with this thing but the idea really isto make expleict what we may already be practising
25
Learning outcomes are explicit statements of what a student is expected to be able to demonstrate as a result of a learning process.
Three domains of learning:
Knowledge
Skills,
Application
Attitudes
What are Learning outcomes?
When learning happens we expect some outcomes and we need to know whether that outcome has been achieved.
Learning outcomes are about been able to demonstrate that achievement in terms of K.S.A ( and their application)
26
Why Learning Outcomes?
Institution(and beyond)
Basis for quality assurance and enhancement
A way of communicating to prospective students, employers and others within the institution about the programme
Facilitates transparency to regulatory bodies and professional organisations
Helps in defining and/or quantifying credit and recognising learning achievement
Facilitates mobility across qualifications and/or levels of study
Students
Clarifies learning expectations
Helps them to monitor their learning progress
Guides them in their preparation for assessment
Lecturers
determines teaching content in terms of the knowledge and skills that students should learn
determines the selection of teaching strategies and the learning activities that are needed to help students learn
directs assessment
forms a basis for evaluating a unit of study
Communication tool between different stakeholders associated with or having a vested interest in learining and teaching
27
Students
Institutions
lecturers
Specific- says exactly what the learner will be able to do
Measurable-can be observed and/or measured( has indicators of performance)
Attainable/Achievable-can be reasonably achieved within a given time and context
Relevant-relevant to the learning needs of the course/unit ( i.e. of value)
Time-framed- should be time bound, within the duration of the course
Characteristics of Good learning outcomes - S.M.A.R.T.
Specific-explicit, unambiguous
Measureable-associated with assessment
Attainable - reasonable, duration-wise(e.g. within a semester);
- also circumstances/mode of study (e.g. use standard chromatography equipment to analyse complex mixtures)
Relevance- important, key outcomes, course fit
Time-framed-at the end of the unit or semester or lesson
28
Course
Broad statements of intention about what the graduates are supposed to demonstrate at the end of the duration of the course.
They are incrementally achieved and are usually highly relational and integrative
Unit
Unit level outcomes have a limited scope are more specific to a particular unit of study and achievable within a much shorter period.
They incrementally build up to the achievement of the course learning outcomes and address a specific topic area or theme of the course
Source: https://www.cpp.edu/~ceis/liberal-studies/program-information/14768852_s.jpg
29
Developing Learning Outcomes
In developing learning outcomes we have to start with the bigger picture, the context within which we develop. Revisiting Brians and Hanas idea, we learnt that:
Every course has a vision of what it wants to achieve in terms of its graduate and this can be captured through learning outcomes.
Generally, in a Bachelors programme,
the final year units learning outcomes directly feed into these learning outcomes
To reach these ultimate outcomes at the final year students need to have achieved foundational and developmental outcomes
So the learning outcomes at these levels should reflect that
This is the idea of backward design; starting with the end in mind (Wiggins and McTighe, )
30
Example of Course Learning Outcome
Bachelor of Key Ring Designing
Analyse societal needs for key rings and implement appropriate plans of actions to meet those particular needs.
Example of Unit Learning outcome
Level 100 unit
Describe the different types of key rings
Level 200 unit
Perform a market survey to establish the demand for the different types of rings
Level 300 unit
Apply project management skills to make key rings for a particular clientele
Using our Bachelor of Key Ring Design we could have an example like this one
31
Example of Course Learning Outcome
Bachelor of Environmental Science
Synthesise and apply appropriate knowledge and evidence from diverse sources to address environmental questions
Apply tools, methods, and theoretical knowledge to solve environmental problems and manage ecosystems effectively
Example of Unit Learning outcome
Level 100
Describe interactions between the lithosphere, biosphere, atmosphere and hydrosphere and how they link to influence ecosystems.
Level 200
Apply a range of environmental monitoring techniques in a field situation
Level 300
At the end of the unit, students should be able to:
Integrate information from case studies with ecological concepts to evaluate wildlife management issues
Course Learning Outcomes are more integrated
Level100; Learning the concepts and describing them
Level 200: Apply some methods to situations
Level 300: integrating theory with practice to address management issue, the sort of things you do in real life
32
Please have a go at drafting some learning outcomes, using the activity sheet.
Activity
COMPONENTS OF A LEARNING OUTCOME
Performance: WHAT the learner is expected to demonstrate, expressed in measurable terms. Compulsory
Conditions: UNDER WHAT CONDITIONS and/or WITHIN WHICH CONTEXT the learners are expected to demonstrate the performance. Not compulsory but desirable
Criteria: HOW WELL the learners are expected to demonstrate the performance i.e. required standard or level. Not compulsory but sometimes necessary.
Write a laboratory report based on their own experimental work, using a standard format of scientific journals.
How to Develop Learning outcomes
Performance
Condition
Criteria
Performance- what a learner must be able to do in measurable or observable terms using an action verb and object of that verb e.g.
Classify organisms in the animal kingdom
Conditions/context- This qualifies the learning outcome further by giving a:
condition within which the performance will happen e.g.
classify organisms in the animal kingdom from given charts and/or specimen
and/or context on which the performance is applied e.g.
classify organisms in the animal kingdom and relate them to their habitats
Criteria-Criteria describe the required level or standard at which the learners are expected to perform. Criteria can be described in terms of accuracy (minimum/maximum), productivity level, time, and degree of excellence or level of detail e.g.
classify organisms in the animal kingdom into vertebrates and invertebrates
While the performance component can adequately describe a learning outcome at other times clarity and specificity is helped by including condition and/or criterion
34
Structure of a learning outcome
Subject matter to be learnt
HOW to demonstrate
the learning
So you can use these components to determining the structure of your learning outcomes.
noting that:
A learning outcome is complete with just the performance component but
That conditions enhances the relevance and focus of the outcome and
Criteria specifies the indicators of performance
35
Blooms taxonomy
Statement of learning, context and criteria
Levels and nature of learning outcomes
36
Bloom Taxonomy (wheel)
Blooms Taxonomy(pyramid)
Developed in the 1950s and revised in in the 1990s Blooms Taxonomy is still the most widely used framework Taxonomy of Educational Objectives .
It describes different levels of cognitive processing.
Revised by Anderson and Krathwohl in 1990s
Each level builds on the previous level, which starts at knowledge, followed by comprehension, application, analysis, synthesis and evaluation. The later three categories are sometimes referred to as "higher order thinking skills," or the skills we might expect University studies to be able to demonstrate.
37
Blooms taxonomy
http://www.nottingham.ac.uk/teaching/documents/guidance/lo-guidance.pdf
Levels and nature of learning outcomes
These are the different levels
38
Through statement of learning, example 1:
Explain the role of marketing in an organisation
Explain the role of marketing in the achievement of an organisations strategic plan
Statement of Learning, Context and Criteria
You can also make a learning outcome more complex or get it to facilitate deep playing around with the statement of learning, context and the criteria. So it is not just the verbs.
SOLO: Structure of The Observed Learning Outcome: a means of classifying learning outcomes according to the their complexity.
Instead of a unistructural learning outcome they suggest relational or integrated learning outcomes especially as students
39
Through statement of learning: example 2
Classify the animal kingdom into vertebrates and invertebrates
Classify the animal kingdom into phyla( and sub phyla) and groups ( and sub groups)
Statement of Learning, Context and criteria
40
Reflect on the levels of your outcomes on the basis of
Blooms Taxonomy
Complexity of the statement of learning
Statement of learning, Context and criteria
41
See Activity Sheet
Review your unit learning outcome on the basis of this structure
Activity
42
Are your Learning outcomes S.M.A.R.T?
Once again
43
Anderson, L. W. and Krathwohl, D. R., et al (Eds.) (2001) A Taxonomy for Learning, Teaching, and Assessing: A Revision of Blooms Taxonomy of Educational Objectives. Allyn & Bacon. Boston, MA (Pearson Education Group).
Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview.Theory into practice,41(4), 212-218.
Wiggins, G. & McTighe, J. (1998). "What is backward design?" InUnderstanding by Design. (1 ed., pp.719). Upper Saddle River, NJ: Merrill Prentice Hall. Retrieved fromhttp://nhlrc.ucla.edu/events/startalkworkshop/readings/backward-design.pd
http://owww.brookes.ac.uk/services/ocsld/resources/writing_learning_outcomes.html
http://www.nottingham.ac.uk/teaching/documents/guidance/lo-guidance.pdf
Bibliography
44
Linking Assessment Tasks to Learning Outcomes
Learning and Teaching Week 01 September 2015
45
Learning outcomes prescribe what students are expected to demonstrate in regards to their learning.
Over the length of the course, both unit level and course level outcomes should be assessed.
Course level assessment tasks are usually assessed in the final year of the course capstone units.
Linking Assessment Tasks to Learning Outcomes
46
If an outcome is not assessed, how will you know if the student has achieved it?
Students need indicators of how they are going within the unit/course which is provided by feedback on assessment tasks not necessarily written feedback.
Assessment Tasks
47
Demonstration of student learning.
Provides direction to the students as to what to focus on.
Enable weighting of assessment tasks to be determined.
To contribute to the achievement of course learning outcomes.
Why Link Assessment Tasks to Learning Outcomes?
48
49
What information do you get from the alignment matrix regarding:
The focus of the unit
The outcomes being assessed
The authenticity of the tasks real life
Productive use of time for academic staff and students
Learning Outcomes and Assessment Tasks
50
Link the task to the world beyond the unit and course authentic tasks.
Make the learning experience worthwhile in its own right.
Are part of a holistic approach to the course, rather than fragmented stand alone activities.
Prompts student self reflection about their own learning.
Ensures a productive use of time for both students and academic staff.
Well Designed Assessment Tasks
51
Clearly communicate the characteristics of the task and criteria for judgment about the task in a way which can not be misinterpreted by students in a different way to that which the academic staff member intended.
Well Designed Assessment Tasks
52
What are the learning outcomes to be assessed?
What are the capabilities or skills (implicit or explicit) in the learning outcomes?
Is the method of assessment able to let students demonstrate the learning outcomes of the unit/course?
Designing an Assessment Task
53
Is the method of assessment efficient in terms of student and academic staff time?
Is the marking scheme/criteria appropriate to the task?
Designing an Assessment Task
54
Using the previous alignment matrix and keeping the same learning outcomes:
What changes or further information would you make to the assessment tasks?
Have a go
55
Biggs, J. and Tang, C. (2007). Teaching for Quality Learning at University. 3rd edition. England and NY: Society for Research into Higher Education and Open University Press.
McIntyre, S. (2007).
Brown, G. (2001). Assessment: A Guide for Lecturers. Learning and Teaching Support Network: Generic Centre. 1-26
http://www.cebe.heacademy.ac.uk/learning/assess/index.php
Evaluating Online Assessment Practice in Art and Design. UNSW Compendium of Good Practice in Learning and Teaching, Issue 5
References
56
Question Time?
57
Today we have discussed the process of developing learning outcomes within any higher education course. The three areas we focused on include:
Course learning outcomes
Unit learning outcomes
Assessments
Summary and Reflections
58
Course learning outcomes
Unit learning outcomes
Assessments
Summary
59
Some points to keep in mind when developing outcomes and assessments!
Consider the big picture when developing unit learning outcomes and assessments
Use of appropriate active verbs in learning outcomes
Ensure active verbs are appropriate to year level
Assessment items explicitly linked to the learning outcomes
Under assessment and over assessment
Reflections
60
200-level Learning Outcome
100-level Learning
Outcome
100-level Learning
Outcome
100-level Learning
Outcome
300-level Learning Outcome300-level Learning OutcomeCourse Learning Outcomes