initial consultation event: timing and publication
TRANSCRIPT
![Page 1: Initial consultation event: Timing and publication](https://reader031.vdocument.in/reader031/viewer/2022030318/5a6589bf7f8b9a931a8b6973/html5/thumbnails/1.jpg)
A feedback survey for taught postgraduatesWorkshop 2: Timing and publication
Catherine Cameron
Senior HE Policy Adviser, HEFCE
Richard Puttock
Head of Data and Management Information, HEFCE
Future Inn, Bristol City Centre
21 September 2017
![Page 2: Initial consultation event: Timing and publication](https://reader031.vdocument.in/reader031/viewer/2022030318/5a6589bf7f8b9a931a8b6973/html5/thumbnails/2.jpg)
Session Outline
1. Survey coverage (10 mins)
2. Survey timing (20 mins)
3. Data linking (10 mins)
4. Publication levels (20 mins)
![Page 3: Initial consultation event: Timing and publication](https://reader031.vdocument.in/reader031/viewer/2022030318/5a6589bf7f8b9a931a8b6973/html5/thumbnails/3.jpg)
Survey coverage
Initial starting point was whether there were any groups of PGT students who should be excluded
Our proposal
All students on PGT courses which lead to a qualification
Excepting those on integrated Masters
Questions or comments?
![Page 4: Initial consultation event: Timing and publication](https://reader031.vdocument.in/reader031/viewer/2022030318/5a6589bf7f8b9a931a8b6973/html5/thumbnails/4.jpg)
Survey timing
Key consideration is to provide representative and meaningful information
Options
• Single survey window
• Post completion survey
• Multi-point survey
• Rolling survey
![Page 5: Initial consultation event: Timing and publication](https://reader031.vdocument.in/reader031/viewer/2022030318/5a6589bf7f8b9a931a8b6973/html5/thumbnails/5.jpg)
Survey timing
End date analysis
Whilst June/July and September/October cover most students, there are differences for:
• Particular providers
• Some students on unstructured courses
• Some students studying particular subjects
End month Percentage of students
Jan 3%
Feb 2%
Mar 3%
Apr 2%
May 4%
Jun 12%
Jul 16%
Aug 5%
Sep 34%
Oct 10%
Nov 6%
Dec 3%
![Page 6: Initial consultation event: Timing and publication](https://reader031.vdocument.in/reader031/viewer/2022030318/5a6589bf7f8b9a931a8b6973/html5/thumbnails/6.jpg)
Survey timing
Conclusion
• A multi-point or rolling survey is the only way to achieve responses which reflect the majority of a students’ academic experience and a sufficiently high response rate.
• Running the survey at four points in the year should be sufficient to capture a significant majority of the PGT population close to the end of their period of study.
![Page 7: Initial consultation event: Timing and publication](https://reader031.vdocument.in/reader031/viewer/2022030318/5a6589bf7f8b9a931a8b6973/html5/thumbnails/7.jpg)
Survey timing - questions
Your questions ….
Our questions
a. What are the challenges for institutions in delivering a multi-point survey?
b. Is there anything we haven’t considered that we should have?
![Page 8: Initial consultation event: Timing and publication](https://reader031.vdocument.in/reader031/viewer/2022030318/5a6589bf7f8b9a931a8b6973/html5/thumbnails/8.jpg)
Data linking
• In order to be able to analyse and represent survey responses by student characteristics we must be able to link back to individualised student data
• This will also allow us to link to other datasets: NSS, Graduate Outcomes for analysis
• Data futures – will introduce in-year data collection and potentially allow us to generate target lists
Questions or comments?
![Page 9: Initial consultation event: Timing and publication](https://reader031.vdocument.in/reader031/viewer/2022030318/5a6589bf7f8b9a931a8b6973/html5/thumbnails/9.jpg)
Publication
• Intend to consult on some elements of publication approach now and some following successful piloting
• The key things that we think that we will need to consider are:
• Publication routes
• Publication thresholds
• Publication level
• Breakdown of responses
• Benchmarking of responses
• Detailed approach to presentation of information
![Page 10: Initial consultation event: Timing and publication](https://reader031.vdocument.in/reader031/viewer/2022030318/5a6589bf7f8b9a931a8b6973/html5/thumbnails/10.jpg)
Publication
Publication routes
Envisage publishing outcomes in similar way to NSS:
• Publishing responses on the Office for Students (OfS) website
• Publishing on the central information source to support decision-making being provided by the Office for Students and the other funding bodies (assuming continuation)
• Publishing an open dataset so that other information providers can make use of this information
• Making responses available to providers through a secure system.
![Page 11: Initial consultation event: Timing and publication](https://reader031.vdocument.in/reader031/viewer/2022030318/5a6589bf7f8b9a931a8b6973/html5/thumbnails/11.jpg)
Publication
Publication thresholds
Propose to adopt same thresholds as used for publication of NSS
• At least 10 students
• 50 per cent of sample
Question
Our analysis is based on these thresholds and we propose not to consult on them. Are there any concerns about this?
![Page 12: Initial consultation event: Timing and publication](https://reader031.vdocument.in/reader031/viewer/2022030318/5a6589bf7f8b9a931a8b6973/html5/thumbnails/12.jpg)
Publication
Potential coverage
However
Expected gaps for some particular types of provision, particularly at FECs, due to very small cohort sizes
Publication level Estimated coverage (70% response rate)
Estimated number of students included
Course 19% 67%
HECoS CAH level 3 57% 94%
HECoS CAH level 2 70% 98%
![Page 13: Initial consultation event: Timing and publication](https://reader031.vdocument.in/reader031/viewer/2022030318/5a6589bf7f8b9a931a8b6973/html5/thumbnails/13.jpg)
Publication
Breakdown of responses
We also need to consider the course characteristics by which the survey responses should be broken down to best meet its purposes
For example, mode of study.
But …. we must be careful not to fragment to an extent where we cannot publish data
Questions
• Are there ways in which is would be essential to break down the data to make it meaningful?
• Are there ways in which it would be desirable to break it down?
![Page 14: Initial consultation event: Timing and publication](https://reader031.vdocument.in/reader031/viewer/2022030318/5a6589bf7f8b9a931a8b6973/html5/thumbnails/14.jpg)
PublicationBenchmarking
• We will consider benchmarking as part of the second stage of developing proposals.
• HEFCE is currently leading a fundamental review of the approach to benchmarking used in the UK HE Performance Indicators and the TEF to ensure it meets best practice.
• This will influence our approach to benchmarking of survey outcomes.
![Page 15: Initial consultation event: Timing and publication](https://reader031.vdocument.in/reader031/viewer/2022030318/5a6589bf7f8b9a931a8b6973/html5/thumbnails/15.jpg)
Other questions or comments?