Intersecting threshold learning outcomes and first year

pedagogy principles: challenges and opportunities

Theda Thomas, Australian Catholic University, Melbourne; Jill Lawrence, University of Southern Queensland; Pamela Allen, University of Tasmania, Hobart,, Joy Wallace, Charles Sturt University, Adrian Jones, Latrobe University, Melbourne, Jennifer Clark, University of New England, Bronwyn Cole and Lynette Sheridan Burns, University of Western Sydney, University of Western Sydney

Overview • TLOs in Global higher education context

– Europe – United States – Australia

• Australian TLO’s investigated • Student engagement and first year

– First year pedagogy principles defined

• Intersections? • Our OLT project

Intended Learning Outcomes One of the key ways in which institutions demonstrate the quality of their educational product is through the specification of Intended Learning Outcomes. The 2007 UNESCO document Quality Assurance and Accreditation a student learning outcome as a ‘statement of what a learner is expected to know, understand, and be able to demonstrate after completion of a process of learning’. a learning outcome includes ‘the specific intellectual and practical skills gained and demonstrated by the successful completion of a unit, course, or program.’ (Vlăsceanu et. al. 2007: 64)

Global context

Europe • Since 1999 Bologna and European Qualifications Framework

(EQF) created a European Higher Education Area (EHEA) to ensure comparable and compatible qualifications and standards across Europe to promote workers' and learners' mobility. – The Bologna Declaration (1999) a joint ministerial

declaration involving 29 European countries (European Higher Education Area)

– The Bologna Process requires European Union member countries to put in place national qualification frameworks to define learning outcomes at various degree levels, with quality assurance systems.’

4

The United Kingdom

• Threshold concepts emerged from a UK national research project into the characteristics of strong teaching and learning environments in the disciplines for undergraduate education (Enhancing Teaching-Learning Environments in Undergraduate Courses, 2005)

• Meyer and Land (2006) and Cousin (2006) certain concepts central to the mastery of their subjects are ‘threshold’ because they have certain features in common.

• The Quality Assurance Agency (QAA) plus discipline bodies established subject-based benchmark statements TLOs required in a course of study

• These defined threshold and typical expectations of a graduate in terms of the abilities and skills needed to develop understanding or competence in the subject.

5

The United States

The National Institute for Learning Outcomes Assessment (NILOA) established in 2008 to “assist institutions and others in discovering and adopting promising practices in the assessment of college student learning outcomes”

• A response to the 2006 Spellings Commission which had raised concerns about higher education standards. – use of assessment data to improve student learning – approaches to public reporting of the data.

• Includes the 300 plus higher education members of the Voluntary System of Accountability

Australia

TLOs stem from the Australian Qualifications Framework (AQF, 2011) provides learning outcomes for different levels of courses and constructs these in terms of minimum discipline knowledge, discipline specific skills and professional capabilities expected of a graduate in a specified discipline area.

7

Professor Iain Hay, Matthew Flinders Distinguished Professor of Human Geography & Australian Discipline Scholar for the Arts, Social Sciences and Humanities

Elements of the regulatory framework.

TEQSA (Tertiary Education Quality & Standards Agency) regulates the higher education sector in line with a standards framework comprising:

Provider standards

Qualification standards

Learning and teaching

standards

Research standards

Information standards

National protocols and ESOS Act

Australian Qualifications Framework (AQF)

Threshold learning

outcomes

Excellence in Research in Australia (ERA)

For the market and regulators

8

Learning and Teaching Academic Standards (LTAS) project.

• In 2010 Government commissioned the Australian Learning and Teaching Council (ALTC) to manage project.

• Purpose to ensure that discipline communities define and take responsibility for implementing academic standards within academic traditions of collegiality, peer review, pre-eminence of disciplines and academic autonomy.

• To facilitate and co-ordinate discipline communities identifying academic standards (‘Disciplines setting standards’).

9

Discipline Groups and Scholars. • Architecture & Design Prof Sue Savage (QUT) • Arts, Social Sciences & Humanities Prof Iain Hay (Flinders) • Building Prof Sidney Newton (UNSW) • Business, Management & Economics A/Prof Mark Freeman (Sydney) • Creative & Performing Arts Prof Jonathan Holmes (UTas) • Education Dr Greg Heath (La Trobe) • Engineering & ICT Prof Ian Cameron (UQ) A/Prof Roger Hadgraft (Melbourne) • Health, Medicine & Vet Science Prof Maree O’Keefe (Adelaide)

Prof Amanda Henderson (Griffith) • Law Prof Mark Israel (UWA)

Prof Sally Kift (QUT) • Science (including Mathematics) Prof Sue Jones (UTas)

Prof Brian Yates (UTas)

10

Professor Iain Hay, Matthew Flinders Distinguished Professor of Human Geography & Australian Discipline Scholar for the Arts, Social Sciences and Humanities

LTAS project intended to help answer questions like:

• What defines a “discipline” graduate in Australia? How does s/he differ from a graduate of History...?

• Does an entry level Australian “nursing graduate” have general capabilities equivalent to a Canadian entry level nurse?

• If I want to employ an Australian “graphic artist” with a Masters degree qualification, what basic capabilities can I be certain s/he has?

• Does an accountant with a degree from an accredited private accountancy school have the same basic knowledge and skills in accountancy as a university graduate?

Project focus was learning standards, not teaching standards.

11

Why was the LTAS project commissioned?

• Federal government agenda to monitor and maintain academic standards in face of increased and broader participation in higher education & broader range of providers.

• Protect “Brand Australia” – global trends, local academic reputations and export education revenue.

• International trends in quality assurance and accreditation (e.g., European Qualifications Framework, QAA Subject Benchmark Statements).

• Facilitate students’ academic and professional mobility.

12

13

Academic standards:

• Academic standards are learning outcomes described in terms of discipline-specific knowledge, skills, and capacity to apply skills and knowledge.

• Expressed as threshold learning outcomes (TLOs) that a graduate of any given discipline (or program) must have achieved.

14

Principles underpinning academic standards’ identification

• Academic standards should be expressed as assessable learning outcomes.

• Input and process (e.g., lab hours) may support, but are not substitutes for, learning outcomes.

• TLOs will ultimately be defined by each discipline community for each level of AQF qualification (e.g., bachelors, masters, doctorate)

• TLOs must be comparable with appropriate international standards (e.g. QAA).

• Should take account of pre-existing professional accreditation standards, where relevant.

15

• Individual institutions may set standards that are over and above the defined thresholds.

• Individual institutions will determine the curriculum, resources, teaching and assessment methods leading to the achievement of TLOs in their institution.

Diversity and academic autonomy to be protected. N.B. This was NOT about a national curriculum.

16

Upon completion of a Bachelor degree with a major in Geography, graduates will be able to:

Knowing 1. Demonstrate a coherent geographical understanding of trends, processes and impacts that shape Australian and other environments and/or societies at different spatial and temporal scales.

2. Demonstrate an understanding of Geography as an academic discipline, including awareness of its concepts, history and principal subfields, whilst acknowledging the contested, provisional and situated nature of geographical understanding.

Thinking 3. Apply geographical thought creatively, critically and appropriately to specific spaces, places and/or environments.

4. Recognise, evaluate and synthesise various views, arguments and sources of knowledge pertinent to solving environmental and social problems.

Investigating and Problem-solving

5. Resolve geographical questions by ethical means, applying evidence-based knowledge and appropriate research techniques, including those associated with field work.

Communicating

6. Communicate geographical perspectives and knowledge effectively to specialist and non-specialist audiences using appropriately selected written, oral and visual means.

Self-directing and collaborating

7. Contribute effectively as a member or leader of diverse teams working in geographical or multidisciplinary contexts.

8. Reflect on and direct their intellectual and professional development as geographers. 17

Standards Future...

• Little more money for standards development? • Heightened expectations that standards be developed and taken account

of in new regulatory environment. But uncertainty about their form, ‘ownership’, implementation, measures…

• Need to forestall perverse approaches (e.g. standardised testing of outcomes)

• Useful to find constructive applications for the standards over and above quality assurance. (‘After standards’) – Program design. – Curriculum development – Promoting Geography: to domestic and international students as well as to

wider community – Discipline ‘stocktaking’ as a foundation for future review

18

First year Experience in higher Education (FYHE)

19

Unites States • By the 1990’s some 350,000 full-time students have been surveyed each year from around 700 colleges and universities across USA by the Higher Education Research Institute at the University of California, Los Angeles (Astin, et al., 1997)

• The National Survey of Student Engagement (NSSE) an instrument designed to measure student engagement and the degree to which institutions provide students with an effective learning environment (Kuh 2001)

20

United Kingdom

The National Student Survey (NSS) (2005) •Aggregate results and cross-institution results and analyses of the NSS are published on the Higher Education Funding Council for England (HEFCE) website. •Detailed results suitable for prospective students are published on the Unistats website, alongside results of the Destination of Leavers from Higher Education survey. •Results from the survey form the student-centred dimension of Teaching Quality Information.

21

Australia

• Australasian Survey of Student Engagement (AUSSE • First conducted with 20 Australian and five New Zealand institutions (2005) •29 institutions 2008 •35 institutions 2009 •Informed Bradley review (2008) and the Australian Government’s subsequent policy reforms and budget response (Australian Government 2009a).

22

Criticisms •Hussey and Smith (2002: 220) LOs ‘misappropriated and adopted widely at all levels within the education system to facilitate the managerial process’ •Frank Furedi (2012) prescriptive nature of LOs diminishes the ‘open-ended nature of the learning experience and fosters a climate that ‘inhibits the capacity of students and teachers to deal with uncertainty’ •Richard Hill (2012: 114, 115) ‘tyranny of learning objectives’ leading to the ‘rigidification of university teaching’. •Much of the ire is attributed to the large scale FYHE surveys, especially in the UK where used as benchmarks.

23

Sally Kift's First Year Pedagogy

6 organising principles of an intentional first year curriculum design transition, diversity, design, engagement, assessment, and evaluation and monitoring.

24

Intersections: TLOs & the first year pedagogy principles

• Largely unexplored in Aust • Both recent but in divergent streams • Focus on different aspects of the student experience

– TLOs measure students’ learning outcomes at the end of the program

– First year principles are how teachers might design curricula and assessment to maximize LOs at the beginning of students’ programs

• Both concerned with quality of curriculum design and assessment, – but only the TLOs officially the subject of quality

assurance processes 25

OLT project: In the Beginning: Revitalising First Year Curriculum.

Investigates what students need to learn in their disciplines in first year in order to help them meet their TLOs in their final year

•identification of the particular skills and standards needed for the first year for each of the disciplines •production of an online toolkit highlighting the skills and examples of good practice in assessments and learning activities that develop those skills •professional development of first year lecturers across a variety of disciplines in the Social Sciences and Humanities •an evaluated approach to benchmarking workshops that could be sustainable across the universities in the DASSH (Deans of Arts, Humanities and Social Sciences) network in the future

26

So Far

•Conducted workshops •Collected exemplars •Cross-reference for first year principles •Work to do

•Any feedback?

27

Reference List APSA (2011). Political Science discipline standards statement. Retrieved on 15th December 2013 from URL: http://www.auspsa.org.au/page/political-science-discipline-standards-statement AQF (2011). Australian Qualifications Framework: First Edition. Retrieved on 21st May 2012 from URL: http://www.aqf.edu.au/Portals/0/Documents/Handbook/ AustQuals%20FrmwrkFirstEditionJuly2011_FINAL.pdf Furedi. F. (2012). The unhappiness principle. Times Higher Education, 29 November. Retrieved on 3rd March 2014 from URL: http://www.timeshighereducation.co.uk/the-unhappiness-principle/421958.article Hay, I. (2010). History learning and teaching academic standards statement. Australian Learning and Teaching Council. Retrieved on 15th December 2013 from URL: http://www.olt.gov.au/system/files/resources/altc_standards_HISTORY_080211_v 2.pdf Hay, I. (2012). Discipline standards in Arts, Social Sciences and Humanities. Retrieved on 15th December 2013 from URL: http://disciplinestandards.pbworks.com/w/page/52831201/Arts%2C%20S ocial%20Sciences%20and%20Humanities Hill, R. (2012). Whackademia: An insider’s account of the troubled university. Sydney: University of New South Wales Press. 28

Hussey, T. & Smith, P. (2002). The trouble with learning outcomes. Active Learning in Higher Education, 3: 220-233 Kift, S. (2009). Articulating a transition pedagogy to scaffold and to enhance the first year student learning experience in Australian higher education. Final Report for ALTC Senior Fellowship Program. Retrieved on 18th December 2013 from URL: http://fyhe.com.au/wp- content/uploads/2012/10/Kift- Sally- ALTC- Senior- Fellowship-Report-Sep-092.pdf Kuh, G. D., Hayek, J. C., Carini, R. M., Ouimet, J. A., Gonyea, R. M., & Kennedy, J. (2001). NSSE technical and norms report. Bloomington, IN: Indiana University Center for Postsecondary Research and Planning. Land, R., Cousin, G., Meyer, J. & Davies, P. (2005). Threshold concepts and troublesome knowledge: implications for course design and evaluation. In Rust, C et al (2005) Improving student learning, diversity and inclusivity, Oxford: Oxford Centre for Staff and Learning Development.

29

30