Michael Jennings

The University of Melbourne 11 May 2015

Michael Jennings, Deb King (Melb), Adam Bridgeman (Syd), Liz Johnson (Deakin)

Not just another diagnostic test!

Michael Jennings

… the Australian Government’s Office of Learning and Teaching.

Today’s talk is brought to you by…

Michael Jennings

B.Engineering Numbers

Yr ’91 ’01 ’02 ’03 ’04 ’05 ’06

No. 400 600 560 530 520 570 750

Yr ’07 ’09 ’10 ’11 ‘12 ’13 ’14

No. 850 970 1000 1100 1200 1200 1200

Table 1. Bachelor of Engineering first-year numbers (UQ enrolment data)

Prereqs:

1991 - English, Maths 1 (Maths B), Maths 2 (Maths C), Physics, Chemistry

1995 onwards - English, Intermediate Maths (Maths B), Physics OR Chemistry

Approx. 70% of students come in with both Inter. & Advanced Maths (B & C)

Michael Jennings

Leo Howard Test – 1972 to1994

Paper test – 2007

Online test – 2009 to present

Science diagnostic test – 2015

History of diagnostic testing at UQ

Michael Jennings

Current Engineering test – 2009 to present

• UQ T&L grant – cross-discipline team of Engineering & Science (inc. Maths)

• Background questions

• F-10 maths & Senior Intermediate Maths

• Physics

• Chemistry

Michael Jennings

Current Engineering test – OLT grant

Led by USQ, involving UQ, UNE, UTS, and UNewcastle

• http://getset.uqcloud.net

20 maths questions–18 MCQs, 2 enter the answer– Can’t remember & never seen

before options

12 physics and 12 chemistry questions

Michael Jennings

Aim of the engineering test

• Is NOT to tell students how much they don’t know / forgotten

• Is to gauge current mathematical / science understanding

• Is to tell students what maths/science is used in their courses so they can choose the appropriate courses – feedback matrix

• Is to identify at-risk students

Michael Jennings

Current Science test – OLT extension grant – 2014/2015

Led by UQ, involving Melbourne, Sydney and Deakin

• https://getset.science.uq.edu.au

19 maths/statistics questions– all MCQ– Can’t remember & never seen

before options

Michael Jennings

Results – engineering test

• Consistent over the years

• Students who have done both Intermediate and Advanced Maths (Maths B & C) at school do better

• Senior topics not done as well as junior ones

• Differentiation and integration

• Correlation between test results, ATAR (OP) and success in first-year engineering

Michael Jennings

Engineering students’ thoughts

Benefit from doing quiz?% response

It gave me insight into the knowledge I need for 1st year 60It flagged some things I have forgotten and need to revise 82

I feel this is the beginning of my learning journey at uni 29

It made me feel that my needs were being assessed 29No, it achieved none of the above 2

Michael Jennings

Are you glad you did it? %

Yes, I have a better idea of what I know and don't know 73

Yes, I have a better understanding of 1st yr knowledge expectations 62

Yes, I feel more confident 26

I’m not glad I did it at all 4

What actions will you take as a result? %

Revise my high school notes 59

Find a relevant text book 40

Ask someone who knows for help 25

Do nothing as I'll pick it up during semester 17

Do nothing because I did well 8

Michael Jennings

What did you dislike about doing the quiz? %

I haven't disliked doing it at all 38

I forgot a lot of what I knew during the holidays 43

It has scared me 11

It was stressful 6

It didn't help 2

It was time consuming and boring 8

I was still in a holiday mood 35

Kavanagh, L., O’Moore, L, & Samuelowicz, K. (2009). Characterising the first year cohort knowledge. Proceedings of the AAEE 2009 conference.

Burton, L., Dowling, D., Kavanagh, L., O’Moore, L, & Wilkes, J. (2012). Examining First Year Students’ Preparedness for Studying Engineering. Proceedings of the AAEE 2012 conference.

Michael Jennings

BSc student’s feedbackSemester 1, 2015

Yes, it gave me an insight into the knowledge that I need for 1st year 60% Yes, it flagged some things that I have forgotten and need to review 79% Yes, I feel that this is the beginning of my journey of studying science at UQ 30% Yes, it made me feel that my needs are being assessed 25% No, it achieved none of the above 3% I'm not glad that I did completed the quiz 8% I have a better idea of what I know and what I don't know 73% I feel more confident 24% I have a better understanding of first year knowledge expectations 58% Revise my high school notes 70% Find a relevant text book 30% Ask someone who knows for help 24% Do nothing because I did well 12% Do nothing because I will pick it up during semester 12% N/A - I didn't dislike it 41% It was stressful 8% It didn't help 4% I forgot a lot of what I knew during the holidays 44% It scared me 16% I was still in a holiday mode 37% It was time consuming and boring 6%

Michael Jennings

2009

Michael Jennings

Results2012

Michael Jennings

Question No ConceptCorrect

Answers%

Q6 Add two numerical fractions 97.54Q7 Add two algebraic fractions 77.05Q8 Solve an equation 92.87Q9 Expand a factorised quadratic 89.47

Q10 Factorise a quadratic 91.95Q11 Solve a quadratic 83.76Q12 Solving simultaneous equations  

  x= 70.04  y= 71.98

Q13 Simplify an expression with powers 76.46Q14 Understanding sine, cos, and tan 84.89Q15 Straight line equations  

  What is the slope of the line? 88.99  What is the y intercept? 78.52

Q16 Simple derivative 92.72Q17 Optimising a function 77.67Q18 Visualising a derivative 65.22Q19 Adding different base logarithms 64.1Q20 Derivatives - Product rule 49.38Q21 Derivatives - Chain rule 73.45Q22 Simple integral 89.66Q23 Definite integral 64.81Q24 Calculate the area under graph 76.6Q25 Composition of functions 64.62

2013

Michael Jennings

Feedback matrix

• https://getset.science.uq.edu.au

Michael Jennings

OLT Extension Grant 2014-2015

• Melbourne, Sydney, Deakin science, maths and business programmes

• UQ Science

• Macquarie

• Southern Cross, Charles Sturt, UWS, Curtin, Griffith, USQ, UTS, UNSW

Michael Jennings

GetSet 2 – what’s new?

Core questions AND options

CORE Option 1

Option 3

Option 2

Michael Jennings

You want GetSet? You got it!

However, you need:

• TIME

• $$

UQ Science 2015

• $43,000 grant – $8000 per discipline

Michael Jennings

What maths is in your course(s)?

What do you want to know? – Procedural vs conceptual understanding.

With what do you think students struggle?

When are you going to do the test?

What are you going to do with the info from the test?

Post-test? End of first-year test? Major test?

QS in Science – www.qsinscience.com.au

Michael Jennings

Acknowledgements

The 2009 online diagnostic test was funded by a The University of Queensland Teaching and Learning grant. The project team was A/Prof Lydia Kavanagh (Faculty of Engineering, Architecture and Information Technology), Dr Liza O’Moore (School of Civil Engineering), Professor Peter Halley (School of Chemical Engineering), Professor Paul Lant (School of Chemical Engineering), Professor Peter Adams (Faculty of Science), Michael Jennings (School of Mathematics and Physics), Professor Lawrie Gahan (School of Chemistry & Molecular Biosciences) and Dr Anton Raynor (School of Mathematics and Physics), .

The OLT funded project involved USQ, UQ, UNE, UTS and UNewcastle. http://www.olt.gov.au/project-get-set-success-using-online-self-assessments-motivate-first-year-engineering-students-engag

The feedback matrix was designed by A/Prof Lydia Kavanagh (Faculty of Engineering, Architecture and Information Technology), Dr Liza O’Moore (School of Civil Engineering).

Michael Jennings

Burton, Lorelle, Dowling, David, Kavanagh, Lydia, O'Moore, Liza & Wilkes, Janelle (2012). Examining first year students' preparedness for studying engineering. In: Llewellyn Mann and Scott Daniel, AAEE 2012 Conference Proceedings. 23rd Annual Conference of the Australasian Association of Engineering Education (AAEE 2012), Melbourne, Australia, (1020-1028). 3 - 5 December 2012.

Jennings, M. (2011). The transition from high school to university: The University of Queensland perspective. In: John Hannah, Mike Thomas and Louise Sheryn, Te Ara Mokoroa: The Long Abiding Path of Knowledge - Proceedings of Volcanic Delta 2011. Volcanic Delta 2011: The Eighth Southern Hemisphere Conference on Teaching and Learning Undergraduate Mathematics and Statistics, Rotorua, New Zealand, (139-149). 27 November - 2 December 2011.

Kavanagh, L., O’Moore, L., & Samuelowicz, K. (2009). Characterising the first year cohort knowledge. 2009 Conference Proceedings. Annual Conference of the Australasian Association of Engineering Education (AAEE 2009).

References