promoting constructive alignment through programme specification and subject benchmarks warren...
TRANSCRIPT
Promoting constructive
alignment through
programme specification and
subject benchmarks
Warren Houghton
School of Engineering and Computer Science,
University of Exeter
Plan.
1. Programme Specification
a. How we went about it in Engineering at Exeter
b. Discussion of process
2. Threshold standards
3. Defining differentiated assessment criteria at module level
– Example and discussion
4. Helping students to manage their own learning
5. Levels of thinking about learning processes
6. A reflective framework for thinking about learning & teaching
Themes
• Responsibility
• Alignment
• Reflection
Context of examples
• University of Exeter
– mid ranking “old” university
– “research lead”
• Department of Engineering
– in School of Engineering and Computer Science
– small general engineering department
– 26 full time academic staff
– approx 400 U/G students
• Heavily constrained by PEI accreditation
Context of examples
3-yrgeneral
BSc
3-yr BEngaccredited
Ele
ctro
nic
Mech
an
ica
l Civ
il
En
g.
&
Man
ag
em
en
t
4-yr MEngaccredited
Ele
ctro
nic
Mech
an
ica
l Civ
il
En
g.
& M
an
Common first year
How we wrote Programme Specifications
1. Put Subject Benchmark Statement to one side !
2. Wrote aims and ILOs for existing programmes
– many iterations - emergent outcomes
3. Wrote aims and ILOs for existing modules
– many iterations
– drawing out what staff were already doing
4. Then, checked against Benchmark Statements etc.
5. Did not try to achieve one-to-one mapping
Why not use Benchmark Statements as “blueprints”?
• What authority should we give the Benchmark
Statements ?
– What does the QAA say ?
• Is there a “correct answer” ?
• How can we obtain a set of required ILOs ?
– From industry?
• Do we have to take responsibility, with our own
ideas?
William Perry’s positions:
1. Absolute right answers are provided by Authority.
2. Authority may make us find his absolute right answers ourselves.
3. Authority may not have found all the absolute right answers – yet . . .
4. Anyone has a right to his own opinion, but better keep Authority happy.
5. Everything is relative.
6. I may have to make some decisions for myself.
7. I must commit myself to a viewpoint.
8. I must take responsibility for what I commit to.
9. I am confident in my personal commitment, but I will keep an open mind.
What position do we take ?What position do we take ?
1. Absolute right answers are provided by Authority.
2. Authority may make us find his absolute right answers ourselves.
3. Authority may not have found all the absolute right answers – yet . . .
4. Anyone has a right to his own opinion, but better keep Authority happy.
5. Everything is relative.
6. I may have to make some decisions for myself.
7. I must commit myself to a viewpoint.
8. I must take responsibility for what I commit to.
9. I am confident in my personal commitment, but I will keep an open mind.
What position are we encouraged to take ?What position are we encouraged to take ?
1. Absolute right answers are provided by Authority.
2. Authority may make us find his absolute right answers ourselves.
3. Authority may not have found all the absolute right answers – yet . . .
4. Anyone has a right to his own opinion, but better keep Authority happy.
5. Everything is relative.
6. I may have to make some decisions for myself.
7. I must commit myself to a viewpoint.
8. I must take responsibility for what I commit to.
9. I am confident in my personal commitment, but I will keep an open mind.
• NOT as some definitive “right answer”
• We have to take responsibility for creating/recreating the curriculum
– drawing on
– our own experience
– others’ experience - set out in benchmarks etc.
• an iterative, reflective, process
How should we use benchmarks?
Curriculum developmentCurriculum development
Emergent Emergent outcomesoutcomes
Rewrite ProgrammeSpecification
Implement changes
Read availableliterature.
Benchmark Statements etc.
Contribute tonational discussion
Compare curriculum with Benchmark Statements etc.
Articulate currentaims, ILOs etc.
Curriculum experienced by teachers and students
A limitation of benchmarks?
• They are not explicitly multidimensional
A two-dimensional table for each assessment criteria heading
Assessment criteria
MEng
BEng
BSc
3rd 2.2 2.1 1st
Breadth and depth
of programme
Degree classification (performance)
MEng and BEng programmes : volumes in a space that is at least three-dimensional.
BEng
MEng
Depth
Breadth
Independence
Dimensions of learning outcomes
Order of thinking
(e.g. Bloom’s hierarchy)
Range of concepts
“difficulty” of concepts
Simplistic “levels” view of a degree
Order of thinking
e.g. Bloom’s
hierarchy
Range of concepts
Allowing for other dimensions
Order of thinking
e.g. Bloom’s
hierarchy
Range or “difficulty” of concepts
and allowing for life before university !
Order of thinking
e.g. Bloom’s
hierarchy
Range or “difficulty” of concepts
Prior learnin
g
Degree programm
e
Plan .
• Programme Specification
– How we went about it in Engineering at Exeter
– Discussion of process
• Threshold standardsThreshold standards
• Defining differentiated assessment criteria at module level
– Example and discussion
• Helping students to manage their own learning
• Levels of thinking about learning processes
• A reflective framework for thinking about learning & teaching
Threshold standards
• benchmarking implies . . .
• all graduates will meet all threshold
standards
• we need to show how
• we may have to change our assessment
• QAA(2000) Engineering Benchmark Statement.
Threshold standards
• Effective in (e.g.) the Royal Navy
• Different in HE – why?
– “Education and training are different”
– Is certification realistic / useable ?
• Too much / different for employers to read
• PDP offers a solution to both problems
Example:
setting and assessing threshold setting and assessing threshold standards in core academic modulesstandards in core academic modules
in Engineering at Exeter
Traditional examination
• 3 hr paper
• Choose 5 out of 8 questions
• Pass mark 40%
• Pass provides evidence of 25% of ILOs tested
• But which 25% ?
• What can we build further learning on ?
• define detailed ILOs/assessment criteria . . .
• For all 1st and 2nd year engineering modules
Module specification – A/B structureDETAILED LEARNING OUTCOMES / ASSESSMENT CRITERIA
List A comprises core outcomes that will be covered fully in lectures and must be achieved by all students to meet the minimum requirement for progression.
List B comprises outcomes that are EITHER more difficult to achieve OR are to be achieved by private study (or both).
A: THRESHOLD LEVEL B: GOOD TO EXCELLENT
.
.Apply nodal analysis, with step by step prompting, to 2 loop circuits....
.
.Apply nodal analysis, without guidance or prompting, to 2 and 3 loop circuits. ..
Assessment - examinations
• Paper A: – Covering list A ILOs only
– Typically, short straightforward questions
– No choice
– Expected mark >80%
– Criteria referenced
• Paper B– All ILOs, and some choice
– Longer, more challenging questions with no “easy” parts
– Expected average < 40%
Doesn’t this approach mean
that we are blatantly
teaching to the examination?
Yes !Yes !
Development
• Accepted because of PEI accreditation
– evidence that students with different marks
had achieved identifiably different learning
outcomes
• Originally developed as part of a
scheme to give better guidance to
students
Impact on staff
• Staff find it hard to split ILOs this way
BUT
• asking for differentiated ILOs seems to
work better than just asking for single
level ILOs
Realism
• Any test is demanding when the pass
mark is 80%
• If we are genuinely going to test all ILOs
they must be achievable.
• We have to be honest.
Deep vs. surface learning
• Are we encouraging surface learning?
• Other factors enable deep learning . . .
• Consider structure of the learning (noun)
– Hierarchy of concepts
• What happens if students try to understand complex concepts when they haven’t grasped the components?
• A/B approach makes deep learning possible
a problem
• of success ?
Supporting students
• A/B split originally introduced for student guidance
• students asked to identify progress against ILOs on weekly basis
• now whole of 1st year
• ILOs are a prerequisite to PDP
Why is PDP important ?
• It is about students becoming autonomous, independent, thinking for themselves– the real purpose of HE
• It enables students to articulate what they can do
PDP:
• is not a bolt on extra
• it is an integral part of learning in HE
• it must be addressed by all academic
teaching staff
Academic staff
• view PDP in qualitatively different ways
and
• many have difficulty with valuing PDP
• Why?
Levels of thinking about teaching
Biggs (1 to 3):
Focus on: 1. what the student is
2. what the teacher does
3. what the student does
plus?
4. how the student can manage 4. how the student can manage
what the student does (PDP) what the student does (PDP)
Ways of thinking about Generic Graduate Attributes
Barrie (1 to 4):
1. Necessary basic PRECURSOR skills but irrelevant as
they are a prerequisite for university entry
2. Useful skills that COMPLEMENT or round out
disciplinary learning
3. These are the abilities that let students TRANSLATE,
make use of or apply disciplinary knowledge in the
world
4. They are the abilities that infuse and ENABLE
university learning and knowledge
Biggs - teachingBiggs - teaching(plus)(plus)
1. What student is
2. What teacher does
3. What student does
4. How student manages learning
1. PRECURSOR, irrelevant
2. Useful COMPLEMENT
3. TRANSLATE learning
4. ENABLE university learning
Barrie - attributes Barrie - attributes (skills+)(skills+)
Paradigm shifts
Plan
Experience
Reflect
TheoriseKolbKolb
Paradigm shifts - double loop learning
New understanding
Paradigm shift
Biggs - teachingBiggs - teaching(plus)(plus)
1. What student is
2. What teacher does
3. What student does
4. How student manages learning
1. PRECURSOR, irrelevant
2. Useful COMPLEMENT
3. TRANSLATE learning
4. ENABLE university learning
Barrie - attributes Barrie - attributes (skills+)(skills+)
Learning to learn
Teachers must :
• have theories of learning
• not just “bags of skills” (see Ramsden)
STUDENTS need
• study “skills”
• AND
• learning about learning
– theories of learning
– tools for metacognition / reflection / self management
A Reflective FrameworkA Reflective Framework
Teacher as creator of subject
learning environment
Teacher as creator of subject
learning environment
Student working within created
learning environment
Student working within created
learning environment
Teacher as provider of tools for reflection on
learning
Teacher as provider of tools for reflection on
learning
Student acquiring tools for reflection
on learning
Student acquiring tools for reflection
on learning
Teacher as learning tutor
Teacher as learning tutor
Student discussing own
learning
Student discussing own
learning
Teacher as subject tutor
Teacher as subject tutor
Student learning through subject related dialogue
Student learning through subject related dialogue
Teacher as reflective
practitioner
Teacher as reflective
practitioner
Student engaged in PDP
Student engaged in PDP
Teacher as creator of subject
learning environment
Teacher as creator of subject
learning environment
Student working within created
learning environment
Student working within created
learning environment
Student engaged in PDP
Student engaged in PDP
The “created learning environment” must be designed so that students can manage their own learning within it.
It must offer:
• real choices for students to make
• resources to support different choices
• information required to make choices (ILOs etc.)
• assessment outcomes clearly linked to choices (i.e. aligned assessment)
P.B.L. ?P.B.L. ?
Teacher as creator of subject
learning environment
Teacher as creator of subject
learning environment
Student working within created
learning environment
Student working within created
learning environment
Teacher as reflective
practitioner
Teacher as reflective
practitioner
Teacher as provider of tools for reflection on
learning
Teacher as provider of tools for reflection on
learning
Student learning about learningStudent learning about learning
Teacher as subject tutor
Teacher as subject tutor
Student learning through subject related dialogue
Student learning through subject related dialogue
Student engaged in PDP
Student engaged in PDP
Teacher as learning tutor
Teacher as learning tutor
Student discussing own
learning
Student discussing own
learning
A dialogue about learning is essential:
Teacher as creator of subject
learning environment
Teacher as creator of subject
learning environment
Student working within created
learning environment
Student working within created
learning environment
Teacher as reflective
practitioner
Teacher as reflective
practitioner
Teacher as provider of tools for reflection on
learning
Teacher as provider of tools for reflection on
learning
Student learning about learningStudent learning about learning
Teacher as subject tutor
Teacher as subject tutor
Student learning through subject related dialogue
Student learning through subject related dialogue
Student engaged in PDP
Student engaged in PDP
Teacher as learning tutor
Teacher as learning tutor
Student discussing own
learning
Student discussing own
learning
PDP as a bolt on extra
PDP:
• should be an integral part of the academic experience
• requires teachers who are reflective practitioners
• should have a profound impact on learning
• can be used as a tool for curriculum development
Plan.
• Programme Specification
– How we did it in Engineering at Exeter
– Discussion of process
• Threshold standards
• Defining differentiated assessment criteria at module level
– Example and discussion
• Helping students to manage their own learning
• Levels of thinking about learning processes
• A reflective framework for thinking about learning & teaching
SummarySummary
Responsibility
Reflection
Alignment
Teacher as creator of subject
learning environment
Teacher as creator of subject
learning environment
Student working within created
learning environment
Student working within created
learning environment
Teacher as reflective
practitioner
Teacher as reflective
practitioner
Teacher as provider of tools for reflection on
learning
Teacher as provider of tools for reflection on
learning
Student learning about learningStudent learning about learning
Teacher as subject tutor
Teacher as subject tutor
Student learning through subject related dialogue
Student learning through subject related dialogue
Teacher as learning tutor
Teacher as learning tutor
Student discussing own
learning
Student discussing own
learning
StudentStudenttakingtaking
responsibilityresponsibility
How do we achieve alignment of:
withHow academic
staff think about research in their
disciplines
How academic staff approach
curriculum development
withHow we want
academic staff to teach
How academic staff are managed
?