Michael Jennings

ACSME 2014

Michael Jennings (UQ), Deb King (Melbourne), Adam Bridgeman (Sydney), Liz Johnson (Latrobe/Deakin)

Not just another diagnostic test!

Michael Jennings

Today’s talk…

…is brought to you by the Office of Learning and Teaching of the Australian Government.

Michael Jennings

B.Engineering Numbers

Yr ’91 ’01 ’02 ’03 ’04 ’05 ’06

No. 400 600 560 530 520 570 750

Yr ’07 ’09 ’10 ’11 ‘12 ’13 ’14

No. 850 970 1000 1100 1200 1200 1200

Bachelor of Engineering first-year numbers (UQ enrolment data)

Prereqs:

1991 - English, Maths 1 (Maths B), Maths 2 (Maths C), Physics, Chemistry

1995 onwards - English, Intermediate Maths (B), Physics OR Chemistry

Approximately 70% of students come in with both Intermediate & Advanced Mathematics (B & C)

Michael Jennings

Diagnostic Test

• Leo Howard Test 1972-1995

• Paper test – 2007

• Online – 2009 to present

History

Michael Jennings

Current test – 2009 to present

• UQ T&L grant – cross-discipline team of Engineering & Science (inc. Maths)

• Background questions

• Learning approaches

• F-10 Maths & Senior Intermediate Maths

• Physics

• Chemistry

Michael Jennings

Current test – OLT grant

Led by USQ, involving UQ, UNE, UTS, and UNewcastle

• http://getset.ceit.uq.edu.au/

20 maths questions–18 MCQs, 2 enter the answer– Can’t remember & never seen

before options

12 physics and 12 chemistry questions

Michael Jennings

Aim of the test

• Is NOT to tell students how much they don’t know / forgotten

• Is to gauge current mathematical / science understanding

• Is to tell students what maths/science is used in their courses so they can choose the appropriate courses

• Is to identify at-risk students

Michael Jennings

Results

• Consistent over the years

• Students who have done both Intermediate and Advanced Mathematics (Maths B & C) at school do better

• Senior topics not done as well as junior ones

• Differentiation and integration

• Correlation between test results, ATAR and success in first-year engineering

Michael Jennings

Students’ thoughts

Benefit from doing quiz?% response

It gave me insight into the knowledge I need for 1st year 60It flagged some things I have forgotten and need to revise 82

I feel this is the beginning of my learning journey at uni 29

It made me feel that my needs were being assessed 29No, it achieved none of the above 2

Michael Jennings

Are you glad you did it? %

Yes, I have a better idea of what I know and don't know 73

Yes, I have a better understanding of 1st yr knowledge expectations 62

Yes, I feel more confident 26

I’m not glad I did it at all 4

What actions will you take as a result? %

Revise my high school notes 59

Find a relevant text book 40

Ask someone who knows for help 25

Do nothing as I'll pick it up during semester 17

Do nothing because I did well 8

Michael Jennings

What did you dislike about doing the quiz? %

I haven't disliked doing it at all 38

I forgot a lot of what I knew during the holidays 43

It has scared me 11

It was stressful 6

It didn't help 2

It was time consuming and boring 8

I was still in a holiday mood 35

Kavanagh, L., O’Moore, L, & Samuelowicz, K. (2009). Characterising the first year cohort knowledge. Proceedings of the AAEE 2009 conference.

Burton, L., Dowling, D., Kavanagh, L., O’Moore, L, & Wilkes, J. (2012). Examining First Year Students’ Preparedness for Studying Engineering. Proceedings of the AAEE 2012 conference.

Michael Jennings

Question No ConceptCorrect

Answers%

Q6 Add two numerical fractions 97.54Q7 Add two algebraic fractions 77.05Q8 Solve an equation 92.87Q9 Expand a factorised quadratic 89.47

Q10 Factorise a quadratic 91.95Q11 Solve a quadratic 83.76Q12 Solving simultaneous equations  

  x= 70.04  y= 71.98

Q13 Simplify an expression with powers 76.46Q14 Understanding sine, cos, and tan 84.89Q15 Straight line equations  

  What is the slope of the line? 88.99  What is the y intercept? 78.52

Q16 Simple derivative 92.72Q17 Optimising a function 77.67Q18 Visualising a derivative 65.22Q19 Adding different base logarithms 64.1Q20 Derivatives - Product rule 49.38Q21 Derivatives - Chain rule 73.45Q22 Simple integral 89.66Q23 Definite integral 64.81Q24 Calculate the area under graph 76.6Q25 Composition of functions 64.62

2013

Michael Jennings

Feedback matrix

• http://getset.ceit.uq.edu.au/

Michael Jennings

OLT Extension Grant 2014-2015

• Melbourne, Sydney, Latrobe science, mathematics and business programmes

• UQ Science

• Macquarie

• Southern Cross, Charles Sturt, Griffith, UWS

Michael Jennings

What maths is in your course(s)?

What do you want to know? – Procedural vs conceptual understanding.

With what do you think students struggle?

When are you going to do the test?

What are you going to do with the info from the test?

QS in Science – www.qsinscience.com.au

Michael Jennings

IT details

• Drupal 7

• Blackboard + other programs

Michael Jennings

Acknowledgements

The 2009 online diagnostic test was funded by a The University of Queensland Teaching and Learning grant. The project team was A/Prof Lydia Kavanagh (Faculty of Engineering, Architecture and Information Technology), Dr Liza O’Moore (School of Civil Engineering), Professor Peter Halley (School of Chemical Engineering), Professor Paul Lant (School of Chemical Engineering), Professor Peter Adams (Faculty of Science), Michael Jennings (School of Mathematics and Physics), Professor Lawrie Gahan (School of Chemistry & Molecular Biosciences) and Dr Anton Raynor (School of Mathematics and Physics), .

The OLT funded project involved USQ, UQ, UNE, UTS and UNewcastle. http://www.olt.gov.au/project-get-set-success-using-online-self-assessments-motivate-first-year-engineering-students-engag

The feedback matrix was designed by A/Prof Lydia Kavanagh (Faculty of Engineering, Architecture and Information Technology), Dr Liza O’Moore (School of Civil Engineering).

Michael Jennings

Burton, Lorelle, Dowling, David, Kavanagh, Lydia, O'Moore, Liza & Wilkes, Janelle (2012). Examining first year students' preparedness for studying engineering. In: Llewellyn Mann and Scott Daniel, AAEE 2012 Conference Proceedings. 23rd Annual Conference of the Australasian Association of Engineering Education (AAEE 2012), Melbourne, Australia, (1020-1028). 3 - 5 December 2012.

Jennings, M. (2011). The transition from high school to university: The University of Queensland perspective. In: John Hannah, Mike Thomas and Louise Sheryn, Te Ara Mokoroa: The Long Abiding Path of Knowledge - Proceedings of Volcanic Delta 2011. Volcanic Delta 2011: The Eighth Southern Hemisphere Conference on Teaching and Learning Undergraduate Mathematics and Statistics, Rotorua, New Zealand, (139-149). 27 November - 2 December 2011.

Kavanagh, L., O’Moore, L., & Samuelowicz, K. (2009). Characterising the first year cohort knowledge. 2009 Conference Proceedings. Annual Conference of the Australasian Association of Engineering Education (AAEE 2009).

References