Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
How to make sure yourstudents learn
what you want them to
Claus Brabrand((( [email protected] )))
Associate Professor((( Programming, Logic, and Semantics )))
IT University of Copenhagen
Claus Brabrand((( [email protected] )))
VISITING Professor((( Software Productivity Group (SPG) )))
Univ. Fed. de Pernambuco
Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
An Introduction to the Theory & Framework of Constructive Alignment
Claus Brabrand((( [email protected] )))
VISITING Professor((( Software Productivity Group (SPG) )))
Univ. Fed. de Pernambuco
[ 3 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
"What is good teaching?"
Exercise:T
[ 4 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Outline (Aug 03, 2010)
1) Introduction Constructive Alignment The SOLO Taxonomy
2) From Content to Competence Advocate a shift in perspective Elaborate on The SOLO Taxonomy
3) In Practice Concrete recommendations Alignment implementation process
[ 5 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
“Teaching for Quality Learning at University
- What the student does”
“Teaching for Quality Learning at University
- What the student does”
Constructive Alignment & SOLO Taxonomy:
Introduction to…
“Teaching Teaching & Understanding Understanding”
“Teaching Teaching & Understanding Understanding”19 min award-winning short-film on Constructive Alignment(available on DVD in 7 languages, epilogue by John Biggs)
John Biggs’ popular and heavily cited book:
Note: 3rd Edition now available [J.Biggs & C.Tang, 2007]
[ 6 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
"What are the film'smain messages
(in your opinion)"?
Neighbour Discussion:T
[ 7 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Outline (Aug 03, 2010)
1) Introduction Constructive Alignment The SOLO Taxonomy
2) From Content to Competence Advocate a shift in perspective Elaborate on The SOLO Taxonomy
3) In Practice Concrete recommendations Alignment implementation process
[ 8 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
From Content to Competence
My old course descriptions (Concurrency 2004): Given in terms of a 'content description'.
Essentially:
This is a bad ideafor two reasons...!
Goal is…:
To understand: deadlock interference synchronization ...
[ 9 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Goal is…:
To understand: deadlock interference synchronization ...
Problem 1 !
Problem with 'content' as goals ! analyze ...theorize ...
define deadlockdescribe solutions
name solutionsrecite conditons
Stud. C
Stud. A
Stud. B
analyze systemsexplain causes
Censor
Teacher
agreem
ent
analyze systemsexplain causes
tacit kno
wled
ge fro
m a
research-b
ased trad
ition
no
t kno
wn
by stu
den
t
[ 10 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Problem 2 !
Problem with 'understanding' as goals !
The answer is simple:
'concept of deadlock' ?!
Goal is…:
To understand: deadlock interference synchronization ...
It cannot be measured !
[ 11 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Competence !
'Competence' as goals !
Have the student do something;and then observe (evaluate) the product and/or process
'SOLO' = Structure of the Observed Learning Outcome
Note': inherently operational (~ verbs)
Objective !
To learn how to: analyze systems for... explain cause/effects... prove properties of... compare methods of... ...
Note: 'understanding' is of course
pre-requisitional !
Competence := knowledge + capacity to act upon it
[ 12 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Neighbour Discussion
"How does thiscontent vs. competence relate to YOUR courses?"
T
[ 13 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
SOLO (elaborated)Note: the list is non-exhaustive
SOLO 2”uni-structural”
SOLO 3“multi-structural”
SOLO 4“relational”
SOLO 5“extended abstract”
theorize generalize hypothesize predict judge reflect transfer theory
(to new domain) …
analyze compare contrast integrate relate explain causes apply theory
(to its domain) …
combine structure describe classify enumerate list do algorithm apply method …
define identify count name recite paraphrase follow (simple)
instructions …
Graphic Legend
problem / question / cue known related issue - given! hypothetical related issue - not given! student response
Q
R
QUANTITATIVE QUALITATIVE
R
R'Q
RQRQRQ
[ 14 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
SOLO verbs
Mapped by: B. Dahl & C. Brabrand
(Natural science context!)
With help from: 3 Educational research
colleagues (medicine) J. Biggs & C. Tang
[ 15 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
SOLO Advantages
Advantages of The SOLO Taxonomy: Linear hierarchical structure (good for progression) Aimed at evaluating student learning Converges on research (at SOLO 5)
Research:Production ofnew knowledge
[ 16 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Exercise
"Key competences in YOUR course?"
T
Concurrency:
analyze systemscompare models
Concurrency:
analyze systemscompare models
Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
10' Break
Please put the Post-Its on the wall
"What is good teaching?" Key competences(in your course)
[ 18 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Outline (Aug 03, 2010)
1) Introduction Constructive Alignment The SOLO Taxonomy
2) From Content to Competence Advocate a shift in perspective Elaborate on The SOLO Taxonomy
3) In Practice Concrete recommendations Alignment implementation process
[ 19 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Concrete Recommendations
Intended Learning Outcomes [Genetics 101]
After the course, the students are expected to be able to: locate genes on chromosomes do simple calculations : (e.g., recombination frequencies,
in-breeding coefficients, Hardy-Weinberg, evolutionary equilibria).
describe and perform connexion-analysis describe fundamental genetic concepts: (e.g., mutation
variation, in-breeding, natural selection). describe and analyze simple inheritancies analyze inheritance of multiple genes simultaneously
2) List sub-goals as 'bullets': Clearer than text
1) Use 'standard formulation':
a) puts learning focus on the student
b) competence formulation: "to be able to"
3) Use 'Verb + Noun' formulation:
What the student is expected to
do with a given matter .V N
V
V
VV
V
V
V
V
N
N
N
N
4) Avoid 'understanding-goals':
"To understand X", "Be familiar with Y", "Have a notion of Z" !
N
[ 20 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Implementation Process
1) Think carefully about: overall goal of course (what students learn to do?)
2) Operationalize these goals and formulate them as SOLO intended learning outcomes
3) Choose carefully the form(s) of assessment (~ intended learning outcomes)
4) Choose carefully the form(s) of teaching (~ intended learning outcomes)
alignmentlearning incentive learning support
Think of teachingactivities as
”training for exam”
[ 21 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
My Starting Point
Content description (Concurrency '04+'05):
What is the overall goal of the course...?(i.e., what are the students to learn to do?)
[ 22 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Overall Course Philosophy
Model-Based Design for Concurrency:
[ 23 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Implementation Process
1) Think carefully about: overall goal of course (what students learn to do?)
2) Operationalize these goals and formulate them as SOLO intended learning outcomes
3) Choose carefully the form(s) of assessment (~ intended learning outcomes)
4) Choose carefully the form(s) of teaching (~ intended learning outcomes)
alignmentlearning incentive learning support
Think of teachingactivities as
”training for exam”
[ 24 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Intended Learning Outcomes
Model-based design
for Concurrency
.. .
SM
I
#1#2
#3
Intended Learning Outcomes(based on The SOLO Taxonomy):
Note:explicitlyincludedas a non-goal
[ 25 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Implementation Process
1) Think carefully about: overall goal of course (what students learn to do?)
2) Operationalize these goals and formulate them as SOLO intended learning outcomes
3) Choose carefully the form(s) of assessment (~ intended learning outcomes)
4) Choose carefully the form(s) of teaching (~ intended learning outcomes)
alignmentlearning incentive learning support
Think of teachingactivities as
”training for exam”
[ 26 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
2004 Multiple-Choice assessment: with a bunch of seemingly reasonable questions:
2006 Assessment (~ ILO's): Project (carefully assessing 'synthesis'):
construct, apply, relate, implement, ... Multiple-Choice test (carefully assessing 'analysis'):
analyze and compare
MC-test (pre- vs. post-alignment)
Bad
Alignment
Good
Alignment
[ 27 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Implementation Process
1) Think carefully about: overall goal of course (what students learn to do?)
2) Operationalize these goals and formulate them as SOLO intended learning outcomes
3) Choose carefully the form(s) of assessment (~ intended learning outcomes)
4) Choose carefully the form(s) of teaching (~ intended learning outcomes)
alignmentlearning incentive learning support
Think of teachingactivities as
”training for exam”
[ 28 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Teaching/Learning Activitiesfunctional knowledge
(problem oriented)
declarative knowledge(discipline oriented)
teacher centric student centric
Lecture
Project work
Exercise class
Case teaching
[ 29 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Lecture (about Application) Teacher activity:
Introduce Explain Elaborate Discuss application Give examples Show PPT slides Questions on slides Winding up
Student activity: Listen Listen (maybe take notes) Understand? (correctly? deeply?) Listen (maybe take notes) Listen (maybe take notes) Watch (maybe note points) Write answers to questions Possibly ask a question
activeteacher
passivestudent
vs.
[ B
igg
s &
Tan
g 2
007
, p
.13
7 ]
vs.
[ 30 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Learn about vs. Learn to do Learning (about):
about application about cooking about programming about designing about analysis about construction about relating ...
Learning (to do): to apply to cook to program to design to analyse to construct to relate ...
studentlistening
studentdoing !!
vs.
(to something about something)
[ 31 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Student Activation
Averageretention rate
5%
10%
20%
30%
50%
75%
80%
Lecture
Reading
Audiovisual
Demonstration
Discussion group
Practice by doing
Teaching others
[ NTL Institute for Applied Behavioral Science, Bethel, Maine ]
passivestudent
activestudent
stud
ent
activ
atio
n
"The (in-famous) Learning Pyramid":
Doing:Learning to doDoing:Learning to do
Listening:Learning aboutListening:Learning about
[ 32 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Make explicit ILO's ( )
(…and tell this to students)
= ILO's = TeachingLearningActivities
Assessment
Constructive Alignment
vs.
ROBERT:extrinsically motivatedROBERT:extrinsically motivated
SUSAN:intrinsically motivatedSUSAN:intrinsically motivated
IntendedLearningOutcomes
[ 33 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Questions...
"What is good teaching?"
The Film
Cognitive processes
Association
new ~ old
The Book
John Biggs
"understanding"
content competence
Student activation
Susan & RobertTeacher models
levels 1 - 2 - 3
Course descriptions
Constructive AlignmentTop Competences
15% programming
CS v. NAT v. MAT
Students at University
My researchand teaching
'TLA'Teaching / Learning
Activities
Tips'n'Tricks
?
recitegeneralize
R
R'
Q
The SOLO Taxonomy
[ 34 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Now, please: "2-minute recap"
Please spend 2' on thinking about and writing down the most important points from the talk – now!:
After 1 dayAfter 1 week
After 3 weeksAfter 2 weeks
Immediately
T
[ 35 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Key References ”Teaching for Quality Learning at University”
John Biggs & Catherine TangSociety for Research into Higher Education, 2007. McGraw-Hill.
”Evaluating the Quality of Learning: The SOLO Taxonomy”John Biggs & Kevin F. CollisLondon: Academic Press, 1982
”Teaching Teaching & Understanding Understanding”Claus Brabrand & Jacob Andersen19 minute award-winning short-film (DVD)Aarhus University Press, Aarhus University, 2006
"Constructive Alignment & The SOLO Taxonomy: a Comparative Study of University Competencies in Computer Science vs. Mathematics"Claus Brabrand & Bettina DahlCRPIT, Vol. 88, ACS 3-17, R. Lister & Simon, Eds., 2007
Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Obrigado !
((( http://www.daimi.au.dk/~brabrand/short-film/ )))
Film's homepage:
[ 37 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Tips'n'Tricks (activation)
Neighbour discussions:
Frequent breaks:
Post-It exercise: focus: zoom in anonymous (!) swap'able everyone will engage empathetic control shared knowledge pool
pu
lse
re
ad
er
me
asu
rem
en
ts:
more questions (students dare ask them)
better questions (students had a chance to discuss)
1-2 min timeout [Phil Race]
Form variation:
lecturing blended with in-class activation exercises
[ 38 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Tips'n'Tricks (cont'd)
"Less-is-more":
Use many examples:(build on student pre-knowledge)
Explicit structure:
analyze compare relate
common deadlock, uncommon deadlock, A-synchronization, B-synchronization, hand-shake, multi-party synchronization, multi-party hand-shake, binary semaphores, generalized semaphores, blocking semaphores, recursive locks, ...
vs.
Emphasize depth over breadth (coverage)
NEWOLD
1. xxxxxxxxxx
2. yyyyyyyyyy
3. zzzzzzzzzz
4. wwwwwww
1. xxxxxxxxxx
2. yyyyyyyyyy
3. zzzzzzzzzz
4. wwwwwww
1. xxxxxxxxxx
2. yyyyyyyyyy
3. zzzzzzzzzz
4. wwwwwww
1. xxxxxxxxxx
2. yyyyyyyyyy
3. zzzzzzzzzz
4. wwwwwww
self evident to you [ teacher ] not to a learner [ student ] (esp. during learning process)
Student 'recap' at end:
after 1 dayafter 1 week
after 3 weeks
after 2 weeks
now
[ 39 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
E.g. course: ”Databases” (at RUC/Roskilde):
Note: almost entirely non-operational(!)
i.e. measure how?!
obtain knowledge about the structure of database systems; be familiar with design of databases by use of special notations like E/R and analysis through normalization; get an overview of the most important database models and a detailed knowledge about the most important model - the relational model as well as the language SQL; get an overview of database indexing and query processing; obtain knowledge about application programming for DB systems.
Problematic Courses
Familiar with ?!
[ 40 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Teacher’sintention
Student’sactivity
Exam’sassessment
e.g.- explain- relate- prove- apply
e.g.- memorize- describe
UNALIGNED COURSE
e.g.- memorize- describe
"Dealing with the test"
[ 41 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Teacher’sintention
Student’sactivity
Exam’sassessment
e.g.- explain- relate- prove- apply
ALIGNED COURSE
e.g.- explain- relate- prove- apply
e.g.- explain- relate- prove- apply
e.g.- explain- relate- prove- apply
e.g.- explain- relate- prove- apply
[ 42 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Student Motivation
Susan: (”intrinsic motivation”) - wants to…: learn ! Robert: (”extrinsic motivation”) - to…: pass exams !
[ 43 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Constructivism
”Transmission is Dead…” : (lectures = ) Knowledge is… Actively Constructed !
active teacher &passive students
!
risk
[ 44 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
SOLO Taxonomy
Hierarchy for Competences:
Deep learning (not surface) !
5: generalize, theorize, predict, …4: explain, analyze, compare, …3: describe, combine, classify, …2: recite, identify, calculate, …
[ 45 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Stud Learning Focus
Focus on Student Learning ! (instead of ”what teacher does” & labelling students: ’good/bad’) Student activitation learning
[ 46 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Alignment
Make explicit ILO’s (Intended Learning Outcomes):
(…and tell this to students)
Exam = ILO’s = Teaching
[ 47 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Teaching/Learning Activities
[kilde: Torben K. Jensen]
problem-oriented
discipline-oriented
teacher-centric student-centric
Lecture
Project work
Exercise class
Case teaching
[ 48 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Acquisition of Competence
Acquisition of competence progresses according to the following stages of learning:
1) Unconscious incompetence 2) Conscious incompetence 3) Conscious competence 4) Unconscious competence
5) Capacity for moving consciously between stages 3) and 4): (which is required by a teacher)
[ 49 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
On the Role of Examination
Alignment: A systemic theory (a teaching system w/ cause/effects) A theory of planning (over the course of a course) A theory of motivation (and incentive)
From the exam as a...:
...to:
"Necessary evil"
Motivational and learning-guidingpedagogical tool for the teacher(!)
applicationof alignment
"The exam does not come after, but before the course!"
[ 50 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
Definition: “Good Teaching”
Definition:
Good news: We now know how to do this:
Alignment!!! Explicitly defined course objectives (as verbs)! Discourage surface-learning! Encourage depth-learning! “Less-is-more”: depth rather than breadth of coverage!
”Good teaching is getting most students to use the higher cognitive level processes that the more academic students use spontaneously”
-- “Teaching for Quality Learning at University”, John Biggs, 2003
[ 51 ]Claus Brabrand Aug 03, 2010Universidade Federal de Pernambuco, Recife, Brazil
The BLOOM Taxonomy (1956)
The BLOOM Taxonomy:
Knowledge
Comprehension
Application
Analysis EvaluationSynthesis
Qualitative
Quantitative
SO
LO 4
+5
SO
LO 2
+3
”[…] really intended to guide the selection of items for a test rather than to evaluate the quality of a student’s response to a particular item”
-- (Biggs & Collis, 1982)”