blitt evaluation data - file · web viewblitt evaluation data - wordpress.com
Post on 30-Jan-2018
217 Views
Preview:
TRANSCRIPT
University of Northampton
BLITT Evaluation Data
Teaching and Learning Research
2015-16
Program Theory for Blended Learning in ITT QTS
1
IMPACT
High quality beginning primary teachers who consistently make use of theory and practice in their teaching
Increased use of technology enhance learning by students
Tutors develop blended teaching and learning resources
Tutors are trained / enabled to use a range of technological tools (used for teaching, learning and assessment)
Tutors have improved knowledge and skills in using technological tools (used for teaching, learning and assessment)
Improved knowledge and understanding of theory and practice (in relation to becoming a primary teacher)
Increased use of technology enhanced teaching by tutors
Blended Learning Program Logic
2
EXTERNAL FACTORS.
Instability in HEI ITT QTS.
University move to Waterside.
University move towards Blended Learning.
New technologies used in teaching and learning.
ASSUMPTIONS
Connections
Provision of blending learning will positively influence knowledge, motivation and participant behaviour.
Tutors engagement improves teaching and learning.
Blended Learning leads to high quality employable beginning teachers.
Tutors use of Blended Learning can lead to recognition within HEA.
ASSUMPTIONS
Pre-conditions
Students and tutors have access to technology
Students and tutors are willing to engage and use technological tools
Effective IT infrastructure to support (NILE)
Tutors value teaching which demonstrates theory and practice
INPUTs
StaffingFundingLearning TechnologistsInfrastructureMaterialsTechnological Resources
OUTPUTS
Programme/Modules produced
Learning Events and Online Learning Package sessions planned and available on NILE
Students enrolled on Programmes/Modules
Blended Learning delivered.
SHORT TERM OUTCOMES
Students participate in Blended Learning
Positive response to resources.
Increased knowledge (for tutors and students) in relation to learning outcomes for modules.
Improved confidence and competence of tutors to use Blended Learning.
Students’ increased skills in using in Blended Learning.
Students report changes in confidence and competence in linking theory and practice.
MEDIUM TERM OUTCOMES
Tutors change T&L strategies to regularly include technology in teaching.
Students’ increased engagement with online learning communities.
Increased links made between theory and practice by students.
Increased links made between theory and practice by tutors.
IMPACT
High quality beginning teachers who consistently make use of theory and practice in their teaching.
TRAINING PROVISION
ITT QTS Staff meetingsCAIROs School ForumCurriculum GroupsLearning TechnologistsPeer to Peer Experts within ITT QTS team – inc apps team
ACTIVITIES
Existing Programmes/Modules reviewedNew T&L strategies developed (Learning Events and Online Learning Packages LE and OLP)New Programme and Module developedStaff meetings organisedTraining designed and deliveredNew T&L resources developedBlended Learning promoted to students
BLITT Project 2015-16- Appropriateness Focus
To what extent is Blended Learning considered suitable in meeting the needs of ITT staff and ITT students.
(BA Primary Education year 1, BA Early Years year 1, PGCE Primary 5-11 and PGCE Primary 3-7) all with QTS.
The responses to questionnaires given to staff and students in December 2015 have been compared in terms of the total % for strongly agree and agree.
Staff BA Primary 5-11 Year One
BA Early Years 3-7 Year One
PGCE Primary 5-11
PGCE Primary 3-7
Cohort Size 15 90 12 41 6Question QuestionI understand what blended learning involves
93 I understand what blended learning involves
83 100 87 100
I understand why blended learning has been introduced
80
The need to introduce blended learning in the University was clearly articulated
67 I understand why the University uses blended learning
76 92 79 83
The need to introduce blended learning in ITT QTS was clearly articulated
87 I understand why teacher training includes blended learning
74 75 88 83
It is not clear which ITT QTS modules will have blended learning in them
80 I am clear which modules include blended learning
69 92 51 50
The theory diagram underpinning blended learning captures the intent of blended learning
87 The diagram shows clearly the intention of blended learning
N/A N/A 69 N/A
The blended learning *LE and OLP, are suited to ITT QTS provision
73 Blended learning* LE and OLP, are an appropriate approach to learning to teach
86 83 72 67
3
The blended learning *LE and OLP, meets University policies on learning and teaching
73 Blended learning *LE and OLP, meets University policies
63 92 69 67
Blended learning meets the needs of the University
93
Blended learning meets the ITT QTS criteria
87
Blended learning meets the needs of teacher educators
53 Blended learning meets the needs of the tutors for teaching trainee teachers
74 83 77 100
Blended learning meets the needs of ITT QTS students
60 Blended learning meets the needs of trainee teachers
77 91 71 83
Blended learning meets the needs of partnership schools
60
Staff Comments (December 2015)
The evidence for and justification of blending learning has not been made convincingly for the University nor ITT QTS. There are some good elements for the LE and 0LP idea but alas some significant drawbacks.
8, 12 & 13 - poor research questions! - Cannot be certain of impact in school life. Overall agree with 8, 12 & 13 but not instances - pure online procedures may/may not be appropriate - I am unsure whereas blended pre-F2F/F2/F/post F2F certainly works well. I strongly believe online procedures will work with TAR but am not convinced with respect to some courses where replaced F2F.
The fact that packages can be restricted can be very beneficial to students. 9 - Unsure of University policies so found this difficult to answer. 10 - Unsure of this question. Would
need to discuss. 14 - Unsure of this question. Would need to consult PSG. 11/12 - Arrangements to secure engagement / assessment not yet in place to ensure impact. 13 - In
theory! There is a lack of openness on the part of the University about the drive to blended learning. It is
financial not pedagogical in its approach. This makes some of the questions above difficult to answer. Over time as staff and students become more familiar with use then impact may become more
apparent. The use of blended learning should not be used as a reason for less face-to-face time with students. - ITT students in particular need this collaborative interaction.
4
BLITT Project 2015-16- Effectiveness Focus
How well has blended learning been implemented to achieve its intended aims.
(BA Primary Education year 1, BA Early Years year 1, PGCE Primary 5-11 and PGCE Primary 3-7) all with QTS.
The responses to questionnaires given to staff in December 2015 have been compared to trainee responses to questionnaires given in December 2015 in terms of the total % responses to
Intended Aims – see Program Theory
Tutors are trained/enabled to use a range of technological tools Tutors have improved knowledge and skills in using technological tools Tutors develop blended teaching and learning resources Increased use of technology enhanced teaching by tutors and increased use of technology enhanced
learning by students Improved knowledge and understanding of theory and practice (in relation to becoming a primary
school teacher)
Staff perceptions on training / being enabled to know and use technology (Dec 2015)
Technology staff supportive but access can be an issue re room in Sulgrave. There have been equipment issues due to multiple department use of room in past. I suggest more support from Technology staff will be required after this at Waterside
Discrepancies and inefficient IT support have seriously impacted upon use of technology, iPads in particular. IT support sporadic. Some IT not Apple TV ready.
IT support have made it very difficult to engage further with IT in sessions. Need Apple TV in all teaching rooms. In early stages so creating e.g. 0LP time consuming.
Time has been allocated for creation. (2) Infrastructure is weak. No video support services for technical issues and editing. (3) Room infrastructure is much weaker. (4) IT support through Belinda is excellent.
As we were rushed into implementing this within modules, time was not used efficiently as training was 'broad-brush' approach rather than tailoring to individuals need as they fuller assessed how to introduce appropriately for their own modules.
Trainees’ perceptions on factors which have prevented their use of blended learning (December 2015).
The responses were coded into themes although these are not independent of each other.
Lack of time/other priorities Navigation Lack of clarity/information Technology issues Other
5
BLITT Project 2015-16- Efficiency Focus
To what extent have the inputs into implementing blended learning been converted into outputs.
(BA Primary Education year 1, BA Early Years year 1, PGCE Primary 5-11 and PGCE Primary 3-7) all with QTS.
The response to questionnaire given to staff in December 2015 in terms of the total % for each of the resources listed below as being used ‘efficiently’ and ‘very efficiently’ to achieve outputs on Program Logic.
Questions %All intended outputs were achieved (Dec 2015)
(with reference to the Program Logic)
53% strongly agree and agree
TIME - time to identify tutor's needs (skills, knowledge) to be able to implement blended learning
86
TIME - time to train individual tutors 60TIME – to train all ITT QTS tutors 60TIME - to access University support 47TIME - to create and load resources on NILE 33RESOURCES – availability of ipads/hardware 20RESOURCES - availability of software 40RESOURCES - availability of IT in teaching rooms 20STAFFING - availability of IT support staff 36STAFFING - availability of Learning Technologists 57STAFFING - availability of peer support within ITT QTS 79BUDGET - availability of funding to support outputs 29
TIME - time to indentify tutor's needs (skills, knowledge) to be able to implement blended learning
TIME - time to train individual totors
TIME - time to train all ITT QTS tutors
TIME - to access University support
TIME - to create and load resources on NILE
RESOURCES - availability of iPads/hardware
RESOURCES - availability of software
RESOURCES - availability of IT in teaching rooms
STAFFING - availability of IT support staff
STAFFING - availability of Learning Technologists
STAFFING - availability of peer support within ITT QTS
BUDGET - availability of funding to support outputs
Production of Learning Events and Online Learning Packages.
6
Responses to monitoring checkpoint Dec 2015 to identify how many and which technological tools had been developed/used.
How many? 0 1-5 6-10 11-15 16+Learning Events developed for yourself
1 8 5 4 9
Learning Events developed for others
3 8 6 3 2
Learning Events used technology in pre F2F
4 9 5 1 3
Learning Events used technology in F2F
5 9 3 5
Learning Events used in post F2F
4 10 5 1 2
Online Learning Packages developed for own modules
7 11 2 1 1
Online Learning Packages developed for others
5 13 4
Technology Tools used by staff – at Dec 2015
a) Most Used (above 6)
Padlet – 14 responses
Kaltura – 9 responses
Video Clips – 7 responses
b) 2- 4 responses for each of these
Google docs, ipads, NILE, YouTube, panopto, ppts, Thinglink, prezi, pic collage, switch, imovie,
c) One response for other tools (generally one or two individual staff only)
Pastoon, google plus, google forms, popplet, book creator, photos, blog + others
Monitoring Implementation Nov 2015
Module Code
Module Title Blended Learning - have we got online learning ?
Blended Learning - have we got Learning Events
ITT1001 Professional Studies 17
ITT1004 Primary Foundation Subjects - 1ITT1005 Subject Specialism 1- HistoryITT1006 Subject Specialism 1- EnglishITT1007 Subject Specialism 1- MathsITT1008 Subject Specialism 1- ScienceITT1009 Subject Specialism 1- PEITT1015 Professional Studies 1- Early YearsITT1016 EYFS: Communication and Language
and LiteracyITT1017 EYFS Areas of Learning
ITT1018 EYFS MathematicsITT1019 Specialism 1: Learning and Teaching
(3-7)ITT1020 Early Years ICT and PedagogyITT1021 Mathematics - 1ITT1022 Art and Design Specialism 1ITT1023 English- 1ITT1024 Science, Design & Technology -1ITT1025 Computing/Technology Enhanced
Learning and RE/PSHE
BLITT Project 2015-16- Impact Focus
What results, intended or unexpected, have been produced directly or indirectly by blended learning.
(BA Primary Education year 1, BA Early Years year 1, PGCE Primary 5-11 and PGCE Primary 3-7) all with QTS.
The responses to questionnaires given to staff in December 2015 and April 2016 have been compared to trainee responses to questionnaires given in December 2015 and April 2016 in terms of the total % responses to ‘completely’ and ‘mostly’.
Intended Short term Outcomes – see Program Logic Students participate in Blended Learning Positive responses to resources Increased knowledge (for tutors and students) in relation to learning outcomes for modules Improved confidence and competence of tutors to use Blended Learning Students’ increased skills in using Blended Learning Students report changes in confidence and competence in linking theory and practice.
Staff BA Primary and BA Early Years
PGCE Primary 5-11 and PGCE Primary 3-7
Cohort 20 99 52QuestionTrainees participate in Blended Learning
80 85 59
Trainees participate in pre F2F activities
60 82 66
Trainees participate in F2F activities
100 99 93
8
Trainees participate in post F2F activities
20 65 31
Trainees respond positively to technological resources used
100 83 87
Trainees have increased knowledge of the learning outcomes of the module
79 90 88
Staff have increased knowledge of the learning outcomes of the module
80
Trainees have increased skills in using blended learning
65 71 73
Staff are more confident in using blended learning
80
Staff are more competent in using blended learning
75
Trainees are more confident in linking theory and practice
55 84 83
Trainees are more competent in linking theory and practice
65 83 85
1. Initial staff perceptions on impact of technology tools on teaching and learning (December 2015)
Teaching and learning
enhances the teaching and learning, enables teaching off site, good ideas for pF2F, F2F and post F2F
a good way to making sessions more visual/classroom based it feels like it has raised the breadth of our teaching styles to enhance student experience just increased variety, impact on learning - not sure means I am aware so can adapt LE as necessary online have helped a lot positive range, opportunities, to merge on/off site, efficiency time some different approaches for students to share ideas, video always good when IT works speed up progression in PE on specific skills they are an essential component of my teaching they have provide variety to the methods of engagement yes, data collection generating ideas, capturing events eg panel discussion for further use in
various ways
Development of resources
easier to access during sessions and for students, however IT services have been most unhelpful
frustration at the provision at times, it takes time to develop the teaching using these, and this has not been available to the right extent
hugh amount of time preparing, relatively short delivery time, very positive but IT has been universally poor
9
lack of reliable IT has made me less confident to use these techniques
Not made a difference
not really undecided - need to get feedback from year 1 trainees I have felt too much under time pressure to try many of the elements Belinda taught us,
although I can see the advantages of being able to use them. Difference to teaching currently, - minimal.
2. Initial Trainee perceptions on benefits from blended learning (December 2015)
Trainees were asked to comment on
How is the use of blended learning approach benefitting your development as a teacher? How is blended learning benefitting the development of your skills in engaging in technology
enhanced learning? What are the benefits to you as a student of blended learning?
Their responses were coded into themes although these are not independent of each other. These initial themes have been identified:
Access to learning –flexibility of being able to access off site Increased knowledge and understanding Reinforcement of learning Linking theory to practice Approaches to learning Resources Organisation Clarity, relevance, workload, timing Not beneficial
3. Staff perceptions on impact of blended learning in supporting trainees to link theory and practice (April 2016).
the links between the 3 components of the BL enable a more structured development, of these links - supporting exploration of them in more depth
better prioritisation of essential information, and then providing opportunities for trainees to apply learning
need to access material on theory prior which is then applied in F2F showing practice
4. Trainee perceptions on impact of blended learning in supporting them to link theory and practice (April 2016). – see full list
UG
Able to link what I have learnt in lesson to practise and understand what to do in certain situations
10
Blended learning reinforces the theory so in a classroom environment background learning allows effective practice
Post-F2F activities allow you to further knowledge and actually have time to reflect on how you would put theory in to practice. Time that you may not yet in the F2F sessions.
The pre-F2F allow for reading around the topic as do the F2F this means when applying and discussing with mentors as trainees it can be seen where it fits practically.
PG
Some of the information online has been useful for better exploring the link between theory and practice.
Blended learning is helpful in my teaching practice, linking technology and teaching- which is paramount in the 21st century.
Research tasks have helped to understand the theory behind certain techniques used within the classroom.
Intended Medium Term Outcomes – see Program Logic
Tutors change T&L strategies to regularly include technology in teaching. Students increased engagement with online learning communities Increased links made between theory and practice by students Increased links made between theory and practice by tutors.
Staff BA Primary and BA Early Years
PGCE Primary 5-11 and 3-1
Cohort 20 99 52QuestionStaff regularly use technology in their teaching
70 92 88
Trainees engage with online learning communities
35 63 54
1. Staff perceptions on how blended learning is supporting trainees to engage with online learning communities.(April 2016)
Increased repertoire of digital tools to support online collaboration increased technological awareness and experience, unintended learning communities being created
2. Trainee perceptions on how blended learning is supporting them to engage with online learning communities. April 2016
The G+ communities have helped develop understanding on a new platform as well as introducing activities for the classroom.
Allows us to learn about different communities - improves knowledge and understanding. Ideas posted on learning communities provide resource/lesson ideas. Helps give resources and ideas of what to input into the online community giving opportunity to
share. It gives you a fantastic bank of learning. We have too many communities to keep up with. It's very
time consuming and keeps you doing work which you may need F2F contact for. It could encourage more people to use online learning communities. For those that do use it, it works
well as it is a place to share ideas that will permanently stay there.
11
BLITT Project 2015-16- Sustainability Focus
Focus Group of 3 staff responses collated
1. What do you think might be the future benefits, if any, from Blended Learning for
a) yourself –
Greater flexibility to my teaching and learning - opportunity to go beyond the here and now e.g using onsite film clips when considering LOTC and more meaningful pre FTF tasks - less sure about some post F2F . At its best when online linked to F2F as opposed to separate can enable more effective use of F2F.
Professional development, evaluation of and reflection on own practice. Developing quality and sustainable models of learning.
b) trainees
Where resource base expanded ( see above) , for tool box types packages - e.g.TAR tailored more effectively to student need.
Taking responsibility individually and collectively for own learning. Raising expectations and challenge.
c) ITT as a whole
Given that so much of our course is offsite, suggest that it ought to be used to link up with these sites to enhance cohesiveness of the course rather reducing out F2F more -we are in a different position to most courses which are essentially university based.
12
Developing communities of enquiry and practice between academic tutors, students and school partners. ·
Less face to face time – double edged: May help trainees to develop greater independence in learning or could just alienate themFocus of sessions can relate more to pedagogy rather than subject knowledge
2. How do you think our Blended Learning approach has been achieved?
A positive ethos in the team coupled with significant opportunities for us to interpret it to our needs Stages to series of activities - my initial fear that it would be an online lecture never came about
although have seen some dreadful stuff on youtube Putting teaching and learning first - including offsite and online Very strong support from Belinda and members of IT team when needed.
Team work and a ‘driver’ in a leadership role. Time allotted to development. A collective commitment to embracing change.
Staged introduction required across whole programmes Staff training Staff meeting time – establish basis and need for Staff time – development of modules
3. How do you think our Blended Learning approach could be used by other areas of the University?
I think the PRE F2F and F2F (some need to refine what we mean post F2F in current model) could be easilyapplied elsewhere.
I think that the models we have created are readily transferable.
Don’ think it should be a one approach fits all
4 What will make our Blended Learning approach sustainable ?
reviewing packages over time - there is a danger that could become ossified knowing how far to go, where it introduce it -using the best balance re student needs - e.g advise
against reducing contact for much of year 1 - strong case for F2F in this year to help transition to HE. developing links with offsite partnerships -schools etc to diversity learning contexts
Continued investment in training and time to develop quality resources approaches collaboratively, and share this good practice. I would really value the opportunity to create DVD footage of teachers in our partnership that we could use in seminars.
Technical support and professionally produced online learning packages Identifying the critical balance of F2F and online to ensure trainees are not put off the course Ensuring expectations on student workload are balanced across the course Ensuring a balance of different types of activity
13
Monitoring of activities and response from tutors to ensure trainees feel work is valued for some designated activities
Clear expectations identified at the outset – trainees clear about their responsibilities
Gathering Evidence for the Blended Learning Study 2015-16
Blended Learning – Initial Evaluation Timescale for BLITT Project
The evaluation framework will use the five themes of Appropriateness, Effectiveness, Efficiency, Impact and Sustainability. The evaluation schedule will be as follows:
Theme December 2015 March 2016 CommentsAppropriateness Students – Questionnaire
See below for C
Staff – QuestionnaireSee below for A
Project Team Focus Group
Student Questionnaires – BA Primary and BA Early Years OnePGCE Primary 5-11 and PGCE Primary 3-7
Staff Questionnaires – all full time and Associate Tutors who teach ITT QTS – ethical issues shared,
Questionnaires – likert and open ended
Project Team Focus Group – Elaine, Jo BB, Anna, Gill C, Paul and Karen L (and Dave)
Effectiveness Students – Questionnaire D and Focus Group
StudentsQuestionnaire
StaffQuestionnaire
Student Questionnaires - BA Primary and BA Early Years Year OnePGCE Primary 5-11 and PGCE Primary 3-7
Student Focus Group – BA Primary Year One – offered to all students to be involved
Efficiency Staff – Questionnaire B StaffQuestionnaire
Staff Focus Group E
Staff Questionnaire – all full time and Associate Tutors who teach on ITT QTS
Staff Focus Group – Gill Chambers, Paul Bracey, Karen Lindley
Impact Students and Staff Questionnaire
Staff Questionnaire – all full time and Associate Tutors who teach on ITT QTS
Student Questionnaire – BA Primary and BA Early Years Year One and PGCE Primary 5-11 and PGCE Primary 3-7
Sustainability Staff Questionnaire
Student Questionnaire
Staff Focus Group
All full time and Associate staff who teach ITT QTS
Student Questionnaire - BA Primary and BA Early Years Year One and PGCE Primary 5-11 and PGCE Primary 3-7
Staff Focus Group – Elaine, Jo BB, Anna, Gill C, Paul, Karen L (and Dave)
14
Research Ethics – confidentiality, anonymous (need to use tool which does not identify person), responses for research questions will be part of the project but the expectation is that staff will be expected to complete the monitoring documents as part of University expectations of monitoring new teaching and learning strategies.
Date Teacher Educator Voice Trainee (Learner) VoiceMarch 25th 2015
Individual feedback via email – to Head of ITT after staff meeting.
June 8th and 9th 2015
Individual reflections and feedback via email to Head of ITT after staff training day on learning theories and technological tools.
Nov 27th 2015
Monitoring use of technological tools – Checkpoint One questionnaire issued during teacher educators meeting.
Focus: Use of technology and Impact on Teaching
Dec 2nd 2015
Staff and Student Meeting – verbal feedback on using technology enhanced tools from first year trainees.
Dec 4th 2015
Blended Learning Questionnaire for first year trainees.
Focus: Appropriateness and Effectiveness
Dec 7th and 8th
Blended Learning Questionnaire for post graduate trainees.
Focus: Appropriateness and Effectiveness
Dec 11th 2015
Staff Questionnaire
Checkpoint Two Focus: Appropriateness
Checkpoint Three – EfficiencyDec 11th 2015
Undergraduate Staff- Student Liaison Committee - feedback on training provision
Jan 6th 2016 Monitoring and development – teacher educator meeting to review module specifications for blended learning and future development hours
March 7th 2016
Undergraduate Staff- Student Liaison Committee - feedback on overall training provision
March 9th 2016
Staff and Student Meeting – verbal feedback on using technology enhanced tools from first year trainees.
March 14th 2016
Postgraduate Staff- Student Liaison Committee – feedback on
15
overall training provisionApril 13th 2015
Staff Questionnaire
Checkpoint Three Focus: Impact
April 22nd 2016
Blended Learning Questionnaire Undergraduates Year One
Focus: ImpactMay 16th 2016
Blended Learning Questionnaire: Postgraduates
Focus: ImpactMay 16th 2016
Staff Project Meeting
Focus: Teaching and Learning PresentationStaff Focus Group Meeting
Focus: Sustainability
There has been a change to the planned use of questionnaires to staff in phase two of the project. The project team reviewed the evidence from phase one and considered if during the time between January and April there would have been any significant differences in teaching using blended learning for trainees and decided there could be limited changes. The two reasons for this decision are firstly in terms of time spent during this time period at University; all postgraduate trainees have been in schools and not attending University Learning Events and the year one trainees have been only attending their specialism modules. Secondly, the data collected in phase one questionnaires provides an insight into perceptions on ‘effectiveness’ which will be followed up with a staff focus group rather than a separate questionnaire as planned.
16
top related