tbl conference – 1 june 2007 1 to adopt or not adopt tbl: a diffusion of innovation perspective...

1

TBL Conference – 1 June 2007

To adopt or not adopt TBL: To adopt or not adopt TBL: a diffusion of innovation perspectivea diffusion of innovation perspective

Mark Freeman & Susan McGrath-Champ

2

Adopt or not?

3

Research aims and motivation

Aim– To better understand the TBL adoption decision

Motivation– Little research on teachers (+ others) experiences– Little research grounded in (innovation) theory– Useful for potential adopters, developers & mgrs

4

Adopt which bits?

5

ContextAustralia

• National research performance exercise• National teaching performance fund

University• On-campus, research-intensive, “Sandstone”• 47,000 commuter students

School (Faculty) Economics and Business • 10,000 students (30% international)• 300 teachers in 11 comprehensive disciplines• Research required and teaching freedom • Transformation since 1999 • Embedded L&T support staff• TBL trial 2006-7

6

Research literature

Innovation in higher education– Typically student experience/tool/method– Teacher experience often peripheral– Nearby peer (Bennett, 01) in ‘tribe’ (Beecher 91)– Other stakeholders (Bell & Bell, 05)

TBL– Approach (Michaelsen et al, 02)– Students enjoyment, engagement, skills, achievement

(Levine et al, 04)– 11 medical teachers in 10 schools (Thompson et al; 07)

7

The “Chasm”

Rogers Diffusion of Innovation

0

10

20

30

40

50

60

70

80

90

100

Innovators2.5%

Early Adopters13.5%

Early Majority34%

Late Majority34%

Laggards 16%

Cum

ulat

ive

%

8

Diffusion of innovation research

Individual factors (eg. Rogers 95,03)– Relative advantage – Cultural compatibility– Trialability– Visibility– Complexity

Non individual factors (Frambach et al, 02)– Internal – External

9

Research method

• Staff interviews – 5 with adopting teachers

• 1 dropped and 1 repeat• Accounting, HR, Ethics

– 4 adoption supporters (TA, Tech, Dev, AMgr)– Questions

• Demographics and context• Model used (Assessment weight? # Quests & quizzes?

Appeals? Problems? Peer assessment?)• 7 open-ended adoption factors

– Transcribed – Categorised by factor

• Student perceptions survey and comparison on marks

10

Relative advantageAdopter+ Quality of learning+ Student experience+ Love instant feedback+ Increase interaction+ Fun factor+ Job satisfaction+ Long term time benefit

- Initial cost (if not supported)- Writing suitable Q- Right teaching space hard

Supporters+ Active learning strategy (AMgr)

+ Less time/$ on marking (TA)

+ The whole TBL model (TA)

- Admin work on quiz & data (TA)

- Technical skill of adopter (Tec)

- Risk of failure (AMgr)

- Might rethink teaching (AMgr)

11

Cultural compatibility

Adopters

+ Paradigm shift to teaching

+ Free to choose teaching style

+ UG students naturally disposed

+ Confident teachers fine

- Few peers interested

- Relinquish control of class

- PG students less comfortable

Supporters+ Fits role of Office of L&T

(Tech)

+ Disrupts dominant ‘telling’ paradigm (Dev)

+ Fits disciplines trialled (AMgr)

+ Fits actual adopters (AMgr)

- Research vs teaching (AMgr)

12

Complexity

Adopters

+ Process straightforward

+ Depends on attitude

- Need to see to believe

- Requires support

Supporters

+Technical staff (Mgr/Dev)

- Varies by % TBL used (AMgr)

- Might rethink teaching (AMgr)

- Technical skill matters (Tec)

- Harder larger class (AMgr)

13

Visibility

Adopters

+ Evident in team quiz results

+ Only if observed in action

+ Students can spread word too

- Inadequate promotion eg. web

Supporters

+ Seeing demystifies (Dev)

+ Hearing peer teaching hum is persuasive (AMgr)

+ Research & conference papers visible (AMgr)

14

Trialability

Adopters

+ Can break into bits

- Need all bits to be truly effective

Supporters

+ Technical support necessary (Tec/Mgr)

+ If interactive Q already used eg. Clickers (AMgr)

- Take a few cycles to refine and get ROI (Dev)

- Big bother for adopting all bits once-off (AMgr)

15

Internal organisation

Adopters+ Extra TA resource+ Enthusiasm & evidence of

developer+ L&T leadership + L&T support

- Inflexibility marking support policy

Supporters+ A budget meant quick

responses possible (Tec)

+ ‘sense of something’ (TA)

+ Strategic positioning under new dean (AMgr)

- Education of adopters (Tec)

16

External environment

Adopters+ Responsive equipment

supplier+ Question banks

- Corrupt publisher question bank

- Prior “TBL” experience

Supporters+ Evidence success

elsewhere (AMgr)

+ Increased rankings pressure (AMgr)

17

Themes

Attributes of sample teachers All student-centred & confident teachers (Thompson, p.254; 2007) All value teaching personally and/or in peer ‘tribe’ All prepared to take some risks

Still face challenges Threshold factors vary and dissipate over time (Thompson, p.253; 2007) Variation apparent with early adopter/mainstream Chasm Course redesign even if pro-interaction and have some Q developed Support stakeholders & context matter lots

Other teachers Not recalcitrant (Newton, p.423; 03)

Just burned, busy, besotted with main reward game

18

Sustainability1. Develop local support systems Appeal to mainstream not early adopter (Anderson et al 98) Online FAQs and templates Online case studies (with video; honest cost/benefits list) Encourage incremental adoption Central equipment supply & maintenance Central JIT development support (both technical & pedagogical)

2. Align local policies Build and allocate appropriate teaching spaces Reward innovation not just teaching quality Reward evaluation and pedagogical research (Anderson et al 98) Grant provisions include collaboration & evaluation report

19

Sustainability

3. Facilitate communities of practice Provide internal opportunities for sharing

• TBLers

• Innovators

• Disciplinary tribe eg. through research seminars

Practice needs to survive champion

20

Conclusion

Case studies of adopter experience important & vary• High impact of formal & informal support structures • High impact of stakeholder’s conception of teaching

Limitations overcome with future research• Ground in other models of diffusion • Bigger, multiple site, unbiased sample• Quantitative• Look for impact of different % of TBL adoption and consider

adopters either side of ‘chasm’• Current http://www.surveymonkey.com/MySurveys.asp?u=768203913765

21

Thank you ….

Q&A

22

Add results: Student perceptions survey

Overall Agree Disagree

Recommend TBL next year 80% 7%

Recommend TBL to other units 65% 13%

More effective learning than other units 48% 19%

23

Add results: Student perceptions survey

Components of TBL Agree Disagree

Regular testing helped progressive learning 81% 7%

In-class IRAT encouraged preparation 99% 1%

IRAT ensures all contribute to team 94% 1%

Prefer do both IRAT and TRAT (not one) 78% 6%

Prefer IRAT not assessable 14% 60%

Prefer TRAT not assessable 7% 72%

24

Add results : Student perceptions survey

Benefits of double testing (IRAT + TRAT) Agree

Increased understanding of concepts 65%

Increased my critical thinking about international HRM 51%

Helped me learn new ways to approach questions 57%

Improved my marks 52%

Other (negotiate, communicate, active) 17%

25

Add results : Student perceptions survey

Challenges of double testing (IRAT + TRAT) Agree

None experienced 38%

Discussion of the team quiz answers 29%

Uneven preparation by team members 26%

Other (talking between teams; teacher’s job to teach; insufficient time)

17%

26

Add results : Student perceptions survey

Components of TBL Agree Disagree

Team RAT developed team skills 80% 7%

Team RAT helped me learn 67% 9%

Team problems helped apply quiz learning 80% 6%

Discussion after TRAT ensures good learn 79% 9%

Team problems too simple to be helpful 20% 63%