testa@greeniwch - institutional approach to improving feedback and assessment practices using testa
TRANSCRIPT
TESTA@GreenwichInstitutional approach to improving feedback and
assessment practices using TESTA at the University of Greenwich
Monika PazioDuncan McKenna
• Nationally and internationally adopted• Mixed methods approach• Programme level assessment
Original TESTA
Scale University of Greenwich % Satisfied
Sector/National % Satisfied
2013 2014 2013 2014
Teaching 86 85 86 87
Assessment and Feedback
73 74 72 72
Academic Support 80 82 80 81
Organisation and Management
78 79 78 78
Learning Resources
85 87 84 85
Personal Development
84 84 82 82
Overall Satisfaction 85 86 85 86
University of Greenwich
• Aim – assessing suitability and effectiveness of TESTA for the UoG context
• March – July 2014• Target – programmes approaching validation• 8 programmes participated, 3 fully completed• Senior management involvement in nomination• Cross university SCA selection, scholarship (5 x £1000)• Followed the original TESTA process
Pilot
Completion rates
Pilot
Research Component
A B C D E F G H
AEQ 80/110students
(71%)
36/112 students
(32%)
26/70Students
(37%)
40/312 students
(13%)
39/245 students
(16%)
50/ 157students(32%)
20/ 92students(22%)
55/ 148students
(37%)
Focus Groups
2(3 and 4 students)
2 (6 and 8 students)
1 (2 students)
1 1 interview(1 x 31 interview 1x1)
The processProgrammes appointed by the DLTs
Initial meeting with the team
questionnaires
Focus groupsaudit
Report writing
Report presentation
• The process• Participant experience• Project team experience• Broader assessment profile• Effectiveness
Evaluation
Evaluation – The process
• Timing • Difficulty is accessing staff and students• Difficulties with focus groups• Lack of engagement with programme
teams –’why me?’ attitude• Low SCA engagement• Patchy workload
Evaluation – The processMethodology
• The questionnaire (the scales, the questions and ‘neutral’)
• The focus on quantity of feedback without acknowledging the importance of quality
- The nature of volume of oral feedback
Evaluation – Participant experience
“it’s very easy, I was worried when I agreed to it thinking what else do
they want me to do but it didn’t require a lot of work from me”
“Also thank you to you and Monica for making the evaluation
process as smooth as it was.”
“The findings provided a useful mapping of assessment
practices that confirmed what we already suspected in a
quantified manner. As such, it helped to inform the
programme level assessment design.”
“Overall we have reduced the number of assessments as well as reducing the reliance on purely written forms.”
• Low Engagement• Money not a long term
motivator • Lack of knowledge of the nature
of the programmeBUT
• Online presence• Resource development• Help with data analysis
Evaluation – Project TeamStudent Change Agents
Evaluation – A&F profile• 33% students directly exposed
(382 participants)• Over 1300 blog views• Over 260 video views• Summer of Innovation entry –
Feed 360(298 votes from 33 institutions)
• difficult to say but….• it makes PL reconsider assessment • triggers discussion about assessment design• provides data to feed into validation documentation• provides evidence for the PL to implement changes
Evaluation - effectiveness
• Programmes contacted 18 months prior to validation
• AEQ - Change to scales and questions, distributed by the staff or SCA
• Audit – Keele approach to audit, additional quality of feedback analysis, verbal vs written feedback ratio
• SCA – recruited from the programme, non-monetary rewards, training programme and resources, personalised approach
• No reports – detailed presentations
TESTA@Greenwich changes to the process
• TESTA accepted as a recommended/ mandatory tool for review
• Recommendations inform changes• Springboard for further projects –
TEL, students as producers
Institutional impact
Institutional implementationSustainability
Centrally managed• Project officer• Responsibility for data
collection on the team• Report• Team providing a service
Centrally supported
• Better cooperation between the team and the PL
• PL participates in the data collection
• Team helping• A presentation
Attributions
http://whowhatwhy.org/wp-content/uploads/2012/12/time-content1.jpg
http://claytrinity.org/wp-content/uploads/2014/07/Road.jpg
https://mackayevangelicalchurch.files.wordpress.com/2011/11/shutterstock_66130165_changes-ahead.jpg?w%3D1024%26h%3D680
http://armstrongeconomics.com/wp-content/uploads/2014/01/Domino-PreventingTheChainReaction.jpg
http://blog.wafa.in/wp-content/uploads/2014/04/How-to-make-checkBoxlist-is-checked-in-yii.jpg
http://upload.wikimedia.org/wikipedia/commons/e/ec/World_map_blank_without_borders.svg
http://monipag.com/mathilde-belledant/wp-content/uploads/sites/1008/MG_5287.jpg
http://www2.gre.ac.uk/__data/assets/thumbnail/0007/443149/gateway-6-4.jpg
http://yhponline.com/wp-content/uploads/2-Business-Intelligence-Tools.jpg
