student survey · 2019-10-21 · 3.3.7 higher order thinking 37 3.3.8 general learning outcomes 39...

STUDENT SURVEY.IE

The Irish Survey of Student Engagement (ISSE)

Implementation of the 2013 National Pilot

AcknowledgementsThe project team wishes to acknowledge the contributionof a large number of people whose assistance wasinvaluable in implementing the pilot national studentsurvey and in developing this report. They include thestudents who participated in pre-testing activities andwho responded to the survey itself; representatives ofthe co-sponsoring organisations; members of the ProjectPlenary Advisory Group and of other project workinggroups; and staff in participating institutions.

The project team wishes to record its gratitude for theinsights and support provided by Professor George D Kuh(Centre for Postsecondary Research, Indiana University)and Associate Professor Hamish Coates (Australian Councilfor Educational Research).

ISSE – 2013 / 01December 2013

The Irish Survey of Student Engagement (ISSE) 2013 1

CONTENTS Executive summary 3

Section 1 Policy context, rationale and project governance 8

1.1 Introduction 8

1.2 Project objectives 9

1.3 Project governance 10

1.4 Why the focus on student engagement and satisfaction? 10

Section 2 Methodology 12

2.1 Design of the survey instrument 12

2.1.1 Overview of the AUSSE 12

2.1.2 Rationale for using the AUSSE as the basis of the ISSE 13

2.2 Target student cohort 14

2.3 Testing the validity and reliability of the ISSE 15

2.3.1 Outcomes from Focus Group and Cognitive Interviews 16

2.4 Reliability of the ISSE 17

Section 3 Results and findings from 2013 national pilot 20

3.1 Introduction 20

3.2 Response rates and representative nature of sample, demographics 20

3.3 Indices and related survey items 21

3.3.1 Academic Challenge 22

3.3.2 Active Learning 25

3.3.3 Student Staff Interactions 27

3.3.4 Enriching Educational Experiences 29

3.3.5 Supportive Learning Environment 32

3.3.6 Work Integrated Learning 35

3.3.7 Higher Order Thinking 37

3.3.8 General Learning Outcomes 39

3.3.9 General Development Outcomes 42

3.3.10 Career Readiness 44

3.3.11 Overall Satisfaction 46


Section 4 National-level analysis of Engagement and Outcomes indices 48

4.1 Introduction 48

4.2 Year/cohort 48

4.3 Institution-type 50

4.4 Mode of study 51

4.5 Programme-type 53

4.6 Field of study 54

4.7 Student characteristics 57

4.7.1 Gender 58

4.7.2 Age 58

4.7.3 Domiciliary of Origin 60

Section 5 National pilot results in the international context 61

5.1 Introduction 61

5.2 ISSE national pilot and AUSSE 2009 and 2012, Undergraduate 62

5.3 ISSE national pilot and POSSE 2012, Postgraduate 63

5.4 International Comparison (Undergraduate) with the US 65

5.5 Conclusion 65

Section 6 Review of the 2013 pilot national survey 66

6.1 Governance and management of the project 66

6.1.1 An effective collaborative partnership 66

6.1.2 Project management and resourcing 66

6.2 Feedback on the survey instrument 67

6.2.1 Feedback from survey participants 67

6.2.2 Feedback from institutions 69

6.3 Communication and promotion 70

6.4 Technical delivery 72

6.5 Timing of fieldwork 72

6.6 Response rates 73

Section 7 ISSE: from pilot to future national surveys 73

7.1 Discussion of the pilot study 73

7.2 Key recommendations for future national student surveys 73

Appendices 75

A.1 Items contributing to each index 76

A.2 Validity and Reliability of the Irish Survey of Student Engagement: Reliability Tables 80

A.3 Participating institutions 86

A.4 Terms of reference for working groups 87

A.5 Membership of project working groups 89

A.6 Fields of study – ISCED explained 91

A.7 Further results from 2013 national pilot 94

A.8 Questions used for pilot 116

A.9 References 127


EXECUTIVE SUMMARYA pilot national survey of students in higher education in Ireland was conducted in 2013. This is the first national survey of student engagement in Ireland and is the first system-wide survey of its kind in Europe. This report describes how the project was implemented and provides an insight into the potential benefits of future national surveys. It also highlights how experiences gained during the pilot phase are being used to inform planning and implementation of future surveys of student engagement.

The project was co-sponsored by the Higher Education Authority (HEA), Institutes of Technology Ireland (IOTI), the Irish Universities Association (IUA) and the Union of Students in Ireland (USI). A unique collaborative partnership structure was put in place to manage, direct and implement the pilot survey project. This national partnership proved highly effective.

This report and further details of the pilot national survey are available at www.studentsurvey.ie

Background and contextThe central objective of this project is to develop a valuable source of information about students’ experiences of higher education in Ireland by asking students themselves. The survey seeks to collect information on how students engage with their learning environments. This information will support institutions to identify practice and provision that are effective and will inform discussion on aspects of existing practice that present particular issues or challenges. The results of the survey are intended to add value, primarily at institutional level, while also informing national discussion and policy.

It is recognised that students have a major contribution to make in the design of curricula, and in reviewing and providing feedback on their experience of college. Good student feedback on engagement and satisfaction will contribute to students experiencing an education that is relevant and responsive to their personal development and growth as fully engaged citizens within society.

The National Strategy for Higher Education to 2030 (DES, 2011) recommended that higher education institutions should put in place systems to capture feedback from students to inform institutional and programme management, as well as national policy. It also recommended that every higher education institution should put in place a comprehensive anonymous student feedback system, coupled with structures to ensure that action is taken promptly in relation to student concerns.

Focus on student engagementThe survey seeks to collect information on student engagement in order to provide a more valuable and informed insight into the experience of students. Engagement with institution (‘college’) life is seen as important in enabling students to develop key capabilities such as critical thinking, problem-solving, writing skills, team work and communication skills. Student engagement is enhanced through involving students in educational processes that enable them to construct their learning and knowledge. Measuring engagement can provide a means to develop a fuller understanding of the student experience above and beyond that ascertained through student satisfaction surveys.


Based on best practice internationallyThe Irish Survey of Student Engagement (ISSE) is based on extensive research conducted in Australia, New Zealand and the US. The National Survey of Student Engagement (NSSE) has been in operation in the US, and beyond, since 2000. The Australasian Survey of Student Engagement (AUSSE) is based on the NSSE but has incorporated additional elements. It has been in operation since 2007 and is increasingly used in Australia and New Zealand. Both of these surveys are designed to measure student engagement. The ISSE is based closely on the AUSSE. Detailed pre-testing was undertaken with students to ensure that the questions used were understood in the Irish national context. The similarities between the three surveys will enable Irish higher education institutions to consider the experiences of their students compared to students in Australasia and the US.

Design of the ISSEStudents are asked over one hundred questions about their experiences of higher education. They answer by selecting the most appropriate response from the options provided. Each of these questions contributes to specific indices relating to student Engagement or Outcomes. Questions offer different numbers of possible responses and different number of questions contribute to each index. This means that, while the indices have a 100 point scale, they cannot be read as simple percentages. The Indices are:

Engagement Indices

— Academic Challenge: the extent to which expectations and assessments challenge students to learn

— Active Learning: students’ efforts to actively construct knowledge

— Student Staff Interactions: the level and nature of students’ contact and interactions with teaching staff

— Enriching Educational Experiences: students’ participation in broadening educational activities

— Supportive Learning Environment: students’ feelings of support within the University (‘college’) community

— Work Integrated Learning: integration of employment-focused work experiences into study

Outcomes Indices

— Higher Order Thinking: participation in higher order forms of thinking

— General Learning Outcomes: development of general competencies

— General Development Outcomes: development of general forms of individual and social development

— Career Readiness: preparation for participation in the professional workforce

— Overall Satisfaction: students’ overall satisfaction with their educational experience

Results of the pilot national survey

A detailed online survey was offered to first year undergraduates, final year undergraduates and postgraduate students on taught programmes. Twenty six institutions participated in the pilot including all Universities, all Institutes of Technology and most Colleges of Education.


More than 12,700 students responded to the pilot national survey which was conducted in March-April 2013. This means that the national results are representative of the overall student voice, even at pilot stage. The project team is very appreciative of the participation of so many students in the 2013 pilot survey. Full details of responses gathered are provided in sections 3 and 4. Some headlines include:

72.4% of all participating students reported positive relationships with teaching staff (5 or greater on 7 point scale leading to ‘available, helpful and sympathetic’)

62.8% of all participating students reported quite a bit or very much when asked if they were provided with the support needed to help them succeed academically

81.5% of all participating students reported positive relationships with other students (5 or greater on 7 point scale leading to friendly, supportive, sense of belonging)

62.3% of all participating students selected often or very often when asked if they were improving knowledge and skills that will contribute to their employability

60.1% of all participating students selected quite a bit or very much when asked if they were acquiring job-related or work-related knowledge and skills

50% of all participating students selected / reported quite a bit or very much when asked if they were solving complex real world problems

60.3% of all participating students selected often or very often when asked if they used an online learning system to complete an assignment

57.8% of all participating students selected often or very often when asked if they had conversations with students of a different ethnicity/nationality

25.3% of all participating students selected plan to or done when asked if they were considering Study abroad/student exchange

79.1% of all participating students selected good or excellent when asked how they would evaluate their entire educational experience at their institutional

84.2% of all participating students selected probably yes or definitely yes when asked if would go to the same institution if they could start all over again

65.3% of all participating students selected often or very often when asked if they applied theories or concepts to practical problems or in new situations

62% of all participating students selected often and very often when asked if they discussed ideas from their coursework with other outside class including students, family member, co-workers etc.

76% of all participating students selected quite a bit or very much when asked if they spend a significant amount of time studying and on academic work.


Students’ reaction to the pilot national surveyA specific question was included in the survey to ask students for their views about the survey itself. Examples of comments from students include:

“The survey was pretty interesting and made me reflect on my own academic year and my performance during classes. Overall it’s a very good survey.”

“I think the survey is a great idea. It is very important to allow the students to voice their opinion and I would appreciate it if the survey is asked to every student before they complete their studies.”

“I think it would be important that the results would be made available not only to University staff but to students as well.”

“I’m happy with this survey so far, and I hope the information provided will actually lead to action, more so than just providing the college with information.”

Testing the validity and reliability of the ISSEA number of procedures were undertaken to test the validity and reliability of the ISSE. These included expert review, focus groups, cognitive interviews and post-fieldwork reliability tests.

Focus groups and cognitive interviews were conducted in four Universities, four Institutes of Technology and one College of Education as part of pre-testing the questions in advance of the national pilot. The vast majority of students found no issues, or only minor issues, when completing or understanding the purpose of the questionnaire. The wording of some individual questions was amended to make them more culturally appropriate to the Irish higher education system. As research postgraduate students reported that the questions were generally not relevant to their experience of higher education, it was decided not to include that cohort in initial implementations of the survey. It is intended to develop an appropriate set of questions to meet the needs of these students in the future.

Post-fieldwork reliability tests demonstrate the overall reliability of the ISSE in the national context. The detail of this testing will inform further development of the survey instrument to maximise its value as a high-quality information source.

More than 12,700 students participated in the pilot survey. This represents a significant response for a pilot. Nevertheless, individual institutions will derive greatest value from the survey when there is a sufficiently high response rate to enable analysis of results for particular sub-groups of the student population within that institution.


Lessons from the pilot national survey Communication and promotion

A series of communications resources, including website, posters and electronic updates, were developed to (i) raise awareness and understanding of the project and (ii) support related activities at institutional level. The website www.studentsurvey.ie was designed as a central reference point for information about the survey and received a significant number of visitors, particularly when the survey was open for participation.

New for 2014

— Direct access to the survey will be provided from the website

— An extensive set of resources will be provided to institutions and to local students’ unions to support promotion at local level

Technical delivery

A number of technical delivery approaches were used during the pilot. Experience of each of the technical solutions contributed to defining the preferred approach for the implementation of future surveys.

New for 2014

— A single technical solution will be used nationally

— A single web link will be used to provide access to the survey. This will be widely promoted.

Timing of fieldwork

Implementation of the pilot survey in March – April 2013 highlighted the variation in calendars between institutions which presented challenges around promotion of the survey in many institutions and is likely to have had a negative impact on overall response rates.

New for 2014

— Institutions will choose the most appropriate three week period during February – March 2014 for local fieldwork, taking account of other events and competing demands on students’ time. This will allow each institution to maximise focussed promotion efforts and to minimise the impact of conflicting activities.

Feedback to students and to staff

As stated at the outset, the central objective of this project is to develop a valuable source of information about students’ experiences of higher education in Ireland. It is critical to the success of the ISSE that significant numbers of students participate. The greatest influence on student participation is the knowledge that the student voice is being listened to.

Aim for 2014

— To increase the awareness amongst staff of the value and benefits of the data generated by the national survey so that they encourage greater student participation

— To ensure that relevant and timely feedback is provided to students and to staff.


SEcTIon 1

POlICY CONTEXT, RATIONAlE ANd PROjECT gOVERNANCE

1.1 IntroductionThe central objective of this project is to develop a valuable source of information about students’ experiences of higher education in Ireland. The survey seeks to collect information on how students engage with their learning environments. This information will support institutions to identify practice and provision that are effective and will inform discussion on aspects of existing practice that present particular issues or challenges. The results of the survey are intended to add value at institutional level primarily by enabling institutional leaders to consider the experiences of different groups of students within that institution. In addition, it will be possible to consider local results in the larger context of similar institution-types, in the overall national context and relative to higher education systems in other countries that have implemented comparable surveys. The greatest value will be derived when there are multiple datasets to consider, facilitating institutions to gauge the impact of any specific initiatives that may have taken place or to identify local trends that may merit further exploration. This is an iterative process with increasing benefits from each additional implementation of the survey and analysis of resulting data. As such, the report of the national pilot points towards the future potential of an additional and valuable instrument to inform institutional planning and national policy. Experiences gained from the pilot, and those of other higher education systems engaged in similar activities, will continue to inform future implementations.

The Universities Act, 1997, and the Qualifications (Education and Training) Act, 1999, make reference to the involvement of students in evaluating the quality of their educational experience. Other key policy influences include Standards and Guidance for Quality Assurance in the European Higher Education Area, (ENQA 2005 and 2009), and Common Principles for Student Involvement in Quality Assurance/Quality Enhancement (IHEQN 2009). These various Acts and reports signal the increasingly accepted importance of involving students in evaluation. The National Strategy for Higher Education to 20301, DES 2011, notes that “substantial progress (in this area) has been made” but also states that “students still lack confidence in the effectiveness of current mechanisms and there remains considerable room for improvement in developing student feedback mechanisms and in closing feedback loops.”

The National Strategy recommended that higher education institutions should put in place systems to capture feedback from students to inform institutional and programme management, as well as national policy. It also recommended that every higher education institution should put in place a comprehensive anonymous student feedback system, coupled with structures to ensure that action is taken promptly in relation to student concerns.

1 http://www.hea.ie/sites/default/files/national_strategy_for_higher_education_2030.pdf


It is recognised that students have a major contribution to make in influencing the design of curricula, and in reviewing and providing feedback on their experience of college. Good student feedback on engagement and satisfaction will contribute to students experiencing an education that is relevant and responsive to their personal development and growth as fully engaged citizens within society.

Until recently, there was no national system in place to explicitly measure the quality of the higher education experience from the perspective of students, although various student surveys were conducted in individual institutions. In spring 2012, a National Academy for Integration of Research, Teaching and Learning (NAIRTL) project to develop a framework for a national student survey concluded its preparatory work. The project report outlined possible implementation mechanisms and made a series of recommendations. Further details are available at www.nairtl.ie/studentsurvey.

Building on the NAIRTL work, a follow up national project was established with a project structure representative of all institutions, relevant agencies and the Union of Students in Ireland. This project undertook to implement a pilot national student survey in March 2013 involving all Universities, Institutes of Technology and Colleges of Education. This project remit included development and implementation of a final survey template, methodology and process for undertaking a more comprehensive national student survey in 2014. The project sought to be inclusive and invited as many student cohorts as possible to participate. It was decided to include postgraduate students in addition to undergraduate students (first and final-years). The outcomes of pre-testing the survey instrument led to a decision to limit postgraduate involvement in the 2013 survey to those students undertaking taught programmes. There is a commitment to develop an appropriate instrument to survey all postgraduate students in the future. Students following different modes of study were also invited to participate in the 2013 survey. Analysis of response rates for different cohorts (full-time, part-time, distance, remote) informed decisions on the cohorts for which results are statistically reliable.

A central project budget was funded by the Higher Education Authority. The project was co-sponsored by the Higher Education Authority (HEA), Institutes of Technology Ireland (IOTI), the Irish Universities Association (IUA) and the Union of Students in Ireland (USI).

1.2 Project objectivesThe objectives for developing and implementing a national student survey were:

— To increase transparency in relation to the student experience in higher education institutions

— To enable direct student input on levels of engagement and satisfaction with their higher education institution

— To identify good practice that enhances the student experience

— To assist institutions to identify issues and challenges affecting the student experience

— To serve as a guide for continual enhancement of institutions’ teaching and learning and student engagement

— To document the experiences of the student population, thus enabling year on year comparisons of key performance indicators

— To facilitate benchmarking with higher education institutions and systems internationally.


1.3 Project governance The governance and management structures for the Irish Survey of Student Engagement (ISSE) were designed to ensure wide representation of partner higher education institutions and sponsoring organisations. A Project Plenary Advisory Group was established with representatives from Universities, Institutes of Technology and the project co-sponsors (HEA, IOTI, IUA and USI). This Plenary Group was responsible for the overall management of the project. There were a number of working groups addressing survey design, communications & reporting, technical issues and data analysis. Each of the sub groups was chaired by a member of the Plenary Group and members were nominated by participating organisations. A full-time project manager was appointed to lead developments and to ensure coherence and consistency between the various elements of the project.

Figure 1. Project management and governance structures

The terms of reference for each group are given in Appendix 4. Membership of working groups is given in Appendix 5.

1.4 Why the focus on student engagement and satisfaction?The Survey Design Group undertook research on practice internationally and determined that a survey focussing on student engagement would most effectively meet the stated objectives for the project.

Engagement with University (‘college’) life is seen as important in enabling students to develop key capabilities such as critical thinking, problem-solving, writing skills, team work and communication skills. Pascarella and Terenzini (2005) demonstrated that an educational environment that stresses student-staff interactions, encourages students to participate in the life of the college, involves students in classroom discussions and is concerned about the academic growth and development of students results in student persistence, degree completion and the increased development of critical thinking skills, analytical competencies, and intellectual development.

Other things being equal, the strongest evidence indicated that the greater the student’s engagement in academic work or in the academic experience of the college, the greater his or her level of knowledge acquisition and general cognitive growth … academic engagement reduced authoritarianism and dogmatism and increased autonomy and independence, intellectual orientation, and the use of principled moral reasoning (Pascarella and Terenzini, 2005).

Survey DesignCommunications

& ReportingTechnical

Data Analysis

Project Plenary Advisory Group

HEA IOTI IUA USI


Through 20 years of research, it has been identified that students who are engaged in the life of the college have a higher quality experience than those at institutions where engagement is not promoted (Kuh, 2001). As Coates (2010) states, ‘contemporary perspectives of student engagement now touch on aspects of teaching, the broader student experience, learners’ lives beyond University, and institutional support’.

Student engagement with higher education is seen as being enhanced through exposing students to a high quality learning environment. Measuring engagement can provide a means to develop a fuller understanding of the student experience above and beyond that ascertained through student satisfaction surveys (Coates, 2010).

Student satisfaction is defined as a measure of the student’s value of the higher education experience (Astin, 1993). A multitude of variables are used to measure students’ satisfaction but generally they are grouped into the broad categories of satisfaction with level of contact and communication with the academic department, satisfaction with college administration, quality of student facilities and overall satisfaction with the higher education experience (Astin 1993, Skilbeck 2001). Student satisfaction as an outcome measure is claimed to be valuable in that, more than any other measure, it is not greatly influenced by the entering characteristics of students, only by the intervention of the higher education environment (Astin 1993).

Based on the research outlined above, and the growing importance in higher education research to evaluate both student engagement and satisfaction, the project undertook to research international practice in order to design an appropriate survey instrument that could be used to measure student engagement and satisfaction across the Irish higher education sector.


SEcTIon 2

METhOdOlOgY2.1 Design of the survey instrument

The Survey Design Group undertook research into international practice and determined that the Irish survey should be based on the extensively-used Australasian Survey of Student Engagement (AUSSE). The rationale for this determination follows.

2.1.1 Overview of the AUSSE

The AUSSE is a suite of instruments that measure student engagement and student outcomes, including satisfaction. The AUSSE was extensively tested prior to use in the Australian higher education system. The AUSSE is based on the National Survey of Student Engagement (NSSE)2, an instrument widely used in the US higher education system. The NSSE was developed as a result of systematic studies of student learning and development linked empirically to student experiences.

National Survey of Student Engagement (US)

The NSSE is based on the premise that ‘what students do during college counts more in terms of desired outcomes than who they are or even where they go to college’ (Kuh, 2000). NSSE measures the extent to which students engage in activities that are related to positive educational outcomes (Kuh, 2000). It was originally developed in the US to be administered to undergraduate students, with a version further developed in the Australasian higher education system to be administered to both undergraduate (AUSSE) and postgraduate (POSSE) students.

The development of the NSSE was based on Chickering and Gamson’s (1987) seven practices in undergraduate education as well as on a number of longstanding instruments that measured the student experience and student outcomes. These included UCLA’s Cooperative Institutional Research Program and Indiana University’s College Student Experiences Questionnaire Research Program (Kuh, Vesper, Connolly, & Pace, 1997; Pace, 1984, 1990).

There is strong support for the NSSE in the US. This can be seen in the substantial growth over the period of a decade in the number of institutions in the US that use the NSSE to understand the student experience. The NSSE was originally piloted in 1999 on 12 institutions and was then rolled out to 56 colleges and Universities in the US (NSSE, 2010). In 2006, a number of Canadian higher education institutions implemented the NSSE. BY 2009, it was estimated that approximately 1,400 US and Canadian higher education institutions had participated in the NSSE (NSSE 2010). The results from the NSSE also receive widespread media coverage in the US including media networks such as ABC, The Washington Post, National Public Radio and USA Today (NSSE 2010). USA Today publishes the results from the NSSE of participating colleges and Universities on a yearly basis.

2 Items used in the 2013 pilot Irish National Student Survey, based on the AUSSE, were licensed through the Centre for Postsecondary Education at Indiana University in the USA who run the National Survey of Student Engagement (NSSE).


Both the NSSE and AUSSE broadly measure the following aspects of the student experiences:

— Exposure to and participation in effective educational practices

— Use of time in and out of class

— What students feel they have gained from their educational experience

— The quality of their interactions with faculty and other students

— The extent to which they feel the institution provides a supportive environment

The AUSSE is now the predominant student outcome measure used in the Australian higher education system.

Six indices are used to measure student engagement, including:

— Academic Challenge - the extent to which expectations and assessments challenge students to learn

— Active Learning - students’ efforts to actively index knowledge

— Student Staff Interactions - the level and nature of students’ contact and interaction with teaching staff

— Enriching Educational Experiences - students’ participation in broadening educational activities

— Supportive Learning Environment - students’ feelings of support within the University community; and

— Work Integrated Learning - integration of employment-focused work experiences into study.

In addition seven outcomes are measured by the AUSSE including:

— Higher Order Thinking - participation in higher-order forms of thinking

— General Learning Outcomes - development of general competencies

— General Development Outcomes - development of general forms of individual and social development

— Career Readiness - preparation for participation in the professional workforce

— Average Overall Grade - average overall grade so far in course

— Departure Intention - non-graduating students’ intentions on not returning to study in the following year

— Overall Satisfaction - students’ overall satisfaction with their educational experience.

Note that Average Overall Grade and Departure Intention are not reported in the Irish survey.

2.1.2 Rationale for using the AUSSE as the basis of the ISSE

— The AUSSE has been extensively developed and tested in the Australian higher education system which has many similarities to the higher education system in Ireland.

— The AUSSE has undergone extensive psychometric testing. This testing has demonstrated the validity and reliability of the instrument as a measure of student engagement and student satisfaction.

— Using the AUSSE as the basis for the ISSE allows higher education institutions in Ireland to benchmark the levels of student engagement and educational outcomes with higher education institutions in Australia, New Zealand, the US and South Africa, countries where the NSSE is now widely used.


— The AUSSE is based on student self-reports of outcomes, such as engagement and satisfaction, which have been identified as valid measures of student outcomes. Furthermore, as the AUSSE is a self-report measure, this gives the student a voice in the evaluative process. In addition, using the student self-report approach of evaluating the impact of an educational programme is attractive from a methodological perspective in that it covers a wide range of learning and developmental outcomes. It is also attractive from a practical perspective since it is ‘fairly inexpensive to survey students and to ask them to report how much they have learned or changed since entering college’ (Anaya, 1999).

— The AUSSE can be analysed at a number of levels, including discipline, gender, national and international students, employment status, full-time/part-time status, mature students, departure intention, distance learning and campus-based students. This allows institutions using the ISSE to identify how various groups undertaking higher education are engaging in, and satisfied with, their experience of higher education.

— The AUSSE has been designed to be administered to the student cohorts identified to be of interest. These include students in the first-year and in the final-year of their undergraduate studies and postgraduate students pursuing taught programmes

— Due to the timelines identified for the development and roll-out of the ISSE, there would have been a number of challenges in developing a completely new survey.

2.2 Target student cohortIt was intended to implement a census survey for all first-year and final-year undergraduate students and for all postgraduate students. Pretesting (Section 2.3) found that the questions accurately captured the experiences of undergraduate students and, with minor amendments, were appropriate to capture the experiences of taught postgraduate students. However, the questions did not fully reflect the experience of research postgraduate students. This finding was similar to the outcomes of pretesting of the AUSSE. The Plenary Group agreed not to include research postgraduate students in the 2013 national pilot. There is a commitment to develop a set of questions to survey the experiences of this cohort in the future. Accordingly, the target student cohort for the 2013 pilot was defined as all first-year and final-year undergraduate students pursuing programmes leading to qualifications included in the National Framework of Qualifications3 (NFQ) at levels 6, 7 and 8; and students pursuing taught postgraduate programmes leading to qualifications included in the NFQ at levels 8, 9 and 104. All modes of study were included (full-time, part-time, distance, e-learning or in-service).

An extract from institutions’ student records systems was used to provide certain limited contextual demographic data which were associated with student responses for high-level analysis. This approach meant that students were not required to input these data when participating in the survey, but that these data could enable analysis of subgroups, for example by demographic and contextual factors such as gender, full-time or part-time, broad field of study. The raw dataset was cleaned in advance of any analysis to remove any fields that could potentially identify any individuals.

3 www.nfq.ie4 A very low number of responses were received from students pursuing studies at NFQ level 10,

leading to these responses being removed from the dataset for statistical reliability.


2.3 Testing the validity and reliability of the ISSEBuilding on the work done in the US on the NSSE (Kuh, 2001) and in Australia on the AUSSE (Coates, 2010), the ISSE underwent a number of pre-testing stages prior to its full use with the target student population in 2013. The rationale for this extensive testing was to ensure that the ISSE is robust in measuring the indices of interest in the Irish higher education system.

Extensive pre-testing of the questionnaires was undertaken through expert review, focus group interviews and cognitive interviews. Non-response or non-completion of questionnaires is a major problem in survey research, leading to the collection of incomplete data, which may affect the reporting of generalised conclusions based on the findings. Questionnaire completion may fail for a number of reasons, including participant non-response, irrelevance of questions or questionnaires to respondents, and inability of respondents to complete questions. A number of methods have been identified in the literature to effectively pre-test questionnaires prior to their distribution, the aim being to ensure that respondents have an understanding of the indices being measured. These methods include focus group interviews and cognitive interviewing (Dillman, 2000, Drennan 2003, 2013).

Focus group and cognitive interviews were completed in nine institutions: four Institutes of Technology, four Universities and one College of Education. Interviews were undertaken with full-time, part-time, national and international and traditional and mature students. All major educational disciplines were represented in the interview process. First-year, final-year and postgraduate students were recruited and invited to either partake in a focus group or cognitive interview by email in each of the respective institutions. In addition, expert review of the questionnaire was undertaken by key stakeholders involved in student surveys in the Irish higher education sector.

Psychometric theory was used in further testing and development of the ISSE. This included classical test statistics (internal consistency and inter-item correlations, factor analysis and item stability). Validity was determined through the use of content- and criterion-related validity tests. Exploratory and confirmatory factor analysis and item response theory were used to test the index validity of the instruments.

The ISSE, following the NSSE, was designed to collect results from students at two points of time in their educational programme: in the second semester of first-year and final-year. The rationale for measuring these two points is that first-year is recognised as a pivotal point in students engaging with higher education; the final-year is the point at which students can look back and evaluate their overall experience of higher education (NSSE 2000). In addition, the ISSE measured the institutional experience of taught postgraduate students. This built on the work done in the Australasian higher education sector with the development of the Postgraduate Survey of Student Engagement (POSSE).

The ISSE was distributed online to students in Institutes of Technology, Universities and Colleges of Education.

Focus Groups

Focus groups were used to generate discussion on whether students from different courses, stage of study and institutions similarly interpreted and understood the questions on the ISSE. Focus groups lasted approximately one hour during which students were asked to review the ISSE and discuss whether there were any difficulties with their responses to the survey. Following the completion of the focus groups, the Survey Design Group reviewed and analysed the responses. The overall outcome of the focus groups was an identification of how students were interpreting the items on the questionnaire and the extent to which this interpretation was consistent across groups of students. It was also used to develop an understanding of specific items that might be problematic as well as ascertaining students’ overall perception of the instrument.


Cognitive Interviewing

Cognitive interviewing is an amalgamation of cognitive theory and survey methodology in the understanding of the processes respondents use in completing survey questionnaires (Drennan 2003, 2013). There has been an exponential growth in the use of the technique as a method of pretesting questionnaires which is based on the think-aloud process. It is through the use of cognitive interviewing that the process a respondent uses to answer a question is understood.

Cognitive interviewing was used as a part of the multistage approach to testing the ISSE. It is a method that is particularly useful when there is uncertainty on how students will answer questions or doubt about their understanding of the wording of questions, especially as both the NSSE and AUSSE, on which the ISSE is based, were developed in different higher education systems. All interviews were audio-recorded with contemporaneous notes taken during the sessions.

2.3.1 Outcomes from Focus Group and Cognitive Interviews

The transcripts and notes from the interviews were analysed by members of the Survey Design Group. Issues identified in the interviews were classified by a framework developed by Conrad et al. (1999) (see Drennan, 2003 for further details). These included the identification of lexical problems (words that respondents had particular problems with), items that were skipped, temporal problems (understanding of response options), logical problems (the ease at which respondents navigated through the questionnaire) and computational problems (for example, problems associated with computing dates, time, number of assignments).

Overall, the vast majority of students identified no issues or only minor issues in completing or understanding the purpose of the questionnaire, indicating that the instrument had both good face and content validity. Some individual wording associated with items that were found to be problematic in the interview process were identified and discussed with the survey design team. This resulted in the minor re-wording of some items to make them culturally appropriate to the higher education system in Ireland.

Students readily accepted the response stems and moved easily between the different time frames, for example, ‘current academic year’ and ‘typical week’. Both undergraduate and taught postgraduate students identified that the ISSE overall was relevant to their programme of study. Taught postgraduate students recommended minor changes to the wording of some items such as the addition of journal articles when measuring engagement with reading. However, research postgraduate students reported, in interviews, that the instrument was generally not relevant to their experience of higher education. This was particularly highlighted in relation to the items that measured classroom-based activities, an area that the majority of research postgraduate students did not have experience of during their doctoral studies. There was a perception from research postgraduate participants that, by the very nature of their studies, they had less contact with academic staff, other students and college societies than students undertaking predominantly taught programmes. It was therefore decided, based on the narrative data collected from research postgraduate students, not to include this cohort in the survey.

As well as feedback on individual items, the focus group and cognitive interviews generated a number of comments from participants on the utility of the survey overall. There was a general consensus that the survey captured both social and academic engagement. Students reported that the value and benefits of the survey needed to be highlighted. They also highlighted the importance of receiving feedback on how the results would be used to improve the student experience. This, it was reported, would encourage students to engage with, and complete, the survey.


Another issue that arose in interviews was the length of the survey; there are approximately one hundred items on the ISSE. Feedback from participants in the focus groups and cognitive interviews highlighted the fact that the survey is comprehensive and covers a lot of detail with respect to academic challenge, active learning, student and staff interactions, the education experience, the learning environment and preparedness for work. There was a perception from participants that the length might dissuade students from completing the survey in a meaningful way, if at all. However, once students engaged with the survey, they reported in interviews that they were able to complete it in a reasonable time. The main emphasis therefore should be on encouraging students to engage with the survey. The perceived length of the survey was not found to be a particular issue once students had completed the questionnaire.

Overall, the layout of the questionnaire was not found by students to be complicated. Items on the ISSE were easy to understand and relevant to students from a variety of years, disciplines and institution-types. There is a need, however, to further test the ISSE with students with disabilities to ensure that the measure of student engagement is accessible to all potential respondents. Participants reported that the items on the ISSE reflected their experience of higher education. Although not included in the pilot survey instrument, participants in the focus group made a number of suggestions on items that may need to be included in future versions of the ISSE. These included items related to student finances, work-life balance, work placements, the impact of social media, and the extent to which students work in groups and teams.

2.4 Reliability of the ISSEThe engagement and outcomes indices that comprise the ISSE are based on the NSSE and AUSSE and therefore use the same metric to measure the student experience. Each index is measured on a 100-point scale that ranges from 0 to 100 with higher results indicating better outcomes in that domain. This allows results on the ISSE to be compared with international student higher education systems as well as across various student cohorts. The scoring will also allow comparisons to be made over time.

Table 2.1 outlines the correlations between the ISSE indices for undergraduate students. Correlations, overall, as would be predicted, were low to moderate suggesting that the indices are not measuring the same domain. The highest correlation was Active Learning and Student Staff Interactions (.46) with the lowest between Supportive Learning Environment and Work Integrated Learning (.18). The correlations between indices measured in the ISSE are in contrast to some other studies where the correlations between the domains were found to be higher. Campbell and Cabrera (2011) reported correlations of greater than .70 between the Active Learning and Student-Faculty Interactions indices and between Enriching Educational Experiences and Active Learning for NSSE. They concluded that correlations of this magnitude might suggest that the indices may only be measuring one or two domains of student engagement rather than the individual indices as claimed, a result not found in the correlations between the indices that comprise the ISSE.

Table 2.1 Correlation of the ISSE Indices (Undergraduate Only)

Index 1 2 3 4 5 6

1. Academic Challenge 1

2. Active Learning .42 1

3. Student Staff Interactions .42 .46 1

4. Enriching Educational Experiences .36 .40 .36 1

5. Supportive Learning Environment .27 .25 .32 .28 1

6. Work Integrated Learning .35 .39 .35 .34 .18 1


Table 2.2 outlines the correlations between the ISSE indices for postgraduate students. Correlations, overall, were similar to those identified for the undergraduate cohort suggesting again that the indices are not measuring the same domain. The highest correlation and lowest correlations, as with undergraduates, was between Active Learning and Student Staff Interactions and Supportive Learning Environment and Work Integrated Learning, respectively.

Table 2.2 Correlation of the ISSE Indices (Postgraduate Only)

Index 1 2 3 4 5 6

1. Academic Challenge 1

2. Active Learning .46 1

3. Student Staff Interactions .38 .48 1

4. Enriching Educational Experiences .28 .38 .34 1

5. Supportive Learning Environment .27 .25 .40 .28 1

6. Work Integrated Learning .29 .33 .22 .32 .10 1

Table 2.3 outlines the internal consistency measures of both the engagement and outcomes indices for the ISSE (first-year and final-year undergraduates and postgraduates). The engagement indices are also compared with the latest internal consistency measures published by NSSE (undergraduate only). Overall the internal consistency measures on the Academic Challenge, Active Learning, Student Staff Interactions and Enriching Educational Experiences indices were slightly below the recommended value of .70. In particular the Cronbach’s alpha measure for Active Learning for first-year respondents was .58. Internationally this scale has had relatively low internal consistency results and this is an issue that will be monitored in further iterations of the ISSE with a view to further development of the Active Learning index.

A number of the Cronbach’s alpha measures on the indices of ISSE were relatively close to the recommended value of .70. Internal consistency measures for first-year, final-year and postgraduate students on the index Supportive Learning Environment and for later undergraduate students on Work integrated Learning were above the recommended value of .70 and these were comparable to values reported on the NSSE. It should be noted that the internal consistency measures on Active Learning and Enriching Educational Experiences published by NSSE were also below .70.

All indices relating to outcomes showed acceptable levels of reliability with Cronbach’s alpha values for first-year, final-year and postgraduate students above the recommended value of 0.70. The outcomes indices are particular to the ISSE and AUSSE; therefore there are no NSSE comparators. Appendix 2 provides further details on the reliability estimates for the engagement and outcomes indices for each institution-type (University, Institute of Technology and College of Education). In addition, Appendix 2 also provides information on the inter-item correlations and corrected item total correlations for both the engagement and outcomes indices.


Table 2.3 Reliability Estimates (Internal Consistency Measures) of the ISSE and NSSE Engagement and Outcomes indices

Index name ISSE (2013) First-year

ISSE (2013) Final-year

ISSE (2013) Postgraduates

nSSE (2011) First-year

nSSE (2011) Final-year

Engagement Indices

Academic Challenge .68 .65 .67 .73 .73

Active Learning .58 .63 .61 .67 .67

Student Staff Interactions .62 .66 .66 .71 .74

Enriching Educational Experiences .65 .67 .68 .60 .66

Supportive Learning Environment .78 .76 .77 .79 .80

Work Integrated Learning* .69 .72 .70 - -

Outcomes Indices

Higher Order Thinking* .76 .77 .80 - -

General Learning Outcomes* .84 .86 .86 - -

General Development Outcomes* .84 .85 .84 - -

Career Readiness* .84 .87 .88 - -

Overall Satisfaction* .77 .81 .83 - -

*These indices are not part of the NSSE.


SEcTIon 3

RESUlTS ANd FINdINgS FROM 2013 NATIONAl PIlOT

3.1 IntroductionThis section presents results from the national pilot of the Irish Survey of Student Engagement (ISSE). It includes an overview of response rates and of the demographic profile of the respondents against that of the relevant student population. This is followed by the national-level results for individual questionnaire items (provided in Appendix 8). Responses to individual questions are presented in groups corresponding to the Engagement or Outcomes indices (Section 2 and Appendix 1). The remaining findings from the survey are presented for the various indices, beginning with national level findings and concluding with Irish scores in an international context.

3.2 Response rates and representative nature of sample, demographicsA total of 12,762 students responded in full to the 2013 survey. This produced an overall response rate of 10.9%. The sample includes 5,577 Undergraduate First Year students, 4,735 Undergraduate Final Year students and 2,450 Postgraduate students. Table 3.1 presents the demographic profile of respondents.

A higher proportion of female students responded than male students (13.5% and 8.4%, respectively) and a higher proportion of students in the Universities responded than in Institutes of Technology (12.5% and 8.9%, respectively). The sample was highly representative of the overall student population in terms of age-group, field of study, mode of study and other key variables. The survey results were weighted (by institution, sex, stage of study and mode of study) to ensure that they are fully representative of the national higher education student population. Responses from non-standard and occasional students were excluded from the sample prior to analysis of the results.

While the overall response rate was relatively low, the sample is substantial and all of the results and findings reported in this publication are statistically robust and valid. In moving to full implementation of the Irish Survey for Student Engagement (ISSE) in 2014, a key objective will be to achieve a significant increase in response rates in order to maximise the value of the survey as a tool for enhancement of teaching and learning within each institution.


Table 3.1 Demographic profile of respondents

Characteristic Population

Responses Response Rate (%)

national 116,642 12,762 10.9%

Age

24 and Under 75,580 65% 8,076 63% 10.7%

25 and Over 40,566 35% 4,612 36% 11.4%

Unknown 496 0% 74 1% 14.9%

Gender

Female 57,372 49% 7,760 61% 13.5%

Male 59,270 51% 5,002 39% 8.4%

Institution-type

University 61,402 53% 7,650 60% 12.5%

Institute of Technology 52,415 45% 4,670 37% 8.9%

College of Education 2,825 2% 442 3% 15.6%

Mode of Study

Full-time 94,677 81% 11,093 87% 11.7%

Part-time 19,623 17% 1,469 12% 7.5%

Remote 2,342 2% 200 2% 8.5%

Field of Studies

General Programmes 242 0% 11 0% 4.5%

Education & Training 6,866 6% 782 6% 11.4%

Arts & Humanities 14,494 12% 1,907 15% 13.2%

Social Science, Business & Law 32,274 28% 3,439 27% 10.7%

Science, Maths & Computing 18,983 16% 2,387 19% 12.6%

Engineering, Manufacturing & Construction 13,629 12% 1,094 9% 8.0%

Agriculture & Veterinary 2,050 2% 251 2% 12.2%

Health & Welfare 17,554 15% 1,835 14% 10.5%

Services 6,126 5% 470 4% 7.7%

Unknown 4,424 4% 586 5% 13.2%

Year/Cohort

Undergraduate – First Year 48,975 42% 5,577 44% 11.4%

Undergraduate – Final Year 44,227 38% 4,735 37% 10.7%

Postgraduate (taught) 23,440 20% 2,450 19% 10.5%

3.3 Indices and related survey itemsThe indices relating to Engagement and Outcomes are most readily understood through an analysis of the component question items. Each question item contributes to specific engagement or outcomes indices. The scores for each index are calculated from responses to multiple questions that contribute to that index. Questions offer different numbers of responses and the number of questions contributing to each index is different. Therefore, for analysis purposes, responses to individual questions were recorded as Likert scales and were aggregated and transformed into the composite index score. Although presented on a scale ranging from 0 to 100, these scores should not be interpreted as percentages of students.

In this section, each of the indices for student engagement or outcomes is presented, followed immediately by the set of individual questions which contribute to that index.


Academic ChallengeAcademic Challenge is defined as the extent to which expectations and assessments challenge students to learn.

Index scores:

By Year of Study By Institution Type By Mode of Study

ISSE UG - Year 1

UG - Final Yr

PGT University IOT Col of Ed Full-time

Part-time

Remote

PE

RC

EN

TIlE

S

Mean 48.3 43.8 50.2 54.9 49.9 45.2 48.8 48.2 48.6 47.9

5th 24.1 21.8 26.5 30.8 26.3 21.0 25.6 24.0 24.6 20.8

25th 38.3 34.4 40.9 45.8 40.0 35.1 40.9 38.2 39.0 38.2

Median 48.3 43.8 50.3 55.2 49.8 44.8 49.0 48.2 48.6 48.5

75th 58.3 52.8 59.8 64.7 60.0 55.3 57.4 58.2 58.7 57.7

95th 72.0 67.0 72.7 76.6 72.7 70.5 71.2 72.1 71.4 72.2

Index scores:

By Age Cohort By domicile group By Sex

23 Years and Under

24 years and over

Unknown Irish Non-Irish Unknown Male Female

PE

RC

EN

TIlE

S

Mean 46.7 50.3 48.0 48.2 51.2 44.7 46.9 49.6

5th 23.3 25.9 24.9 24.0 26.1 24.4 23.3 25.5

25th 36.7 40.7 36.9 38.2 40.8 33.9 36.9 39.7

Median 46.8 50.4 50.5 48.3 51.0 44.0 46.5 49.8

75th 56.5 60.7 55.8 58.2 61.7 53.7 56.6 59.7

95th 70.8 73.5 68.8 71.9 76.6 68.5 70.9 72.6

0

20

40

60

80

100

ISSE UGYear 1

UGFinal Year

University IOT Col of Ed Full Time Part Time RemotePGT

0

20

40

60

80

100

23 Years & Under

24 Years & Over


0

20

40

60

80

100

95th

75th

Median

25th

5th

Guide to Index Scores


Academic ChallengeIndex scores:

By Field of Study

Education & Training

Arts & Humanities

Social Sci, Bus & Law

Sc, Maths & Comp

Eng, Manuf &

Const

Ag & Vet Health & Welfare

ServicesP

ER

CE

NTI

lES

Mean 51.7 49.6 49.3 45.7 46.0 45.2 49.7 43.7

5th 29.2 25.8 25.0 21.3 21.8 23.3 26.3 19.6

25th 43.5 40.7 39.2 35.6 36.2 33.8 39.5 33.7

Median 52.0 49.6 49.5 45.6 45.6 44.0 49.6 42.2

75th 60.7 59.2 59.4 56.0 55.3 55.4 60.1 54.7

95th 71.9 71.5 72.6 71.3 70.3 69.9 73.5 69.0

0

20

40

60

80

100

Education& Training

Arts &Humanities


Sci, Maths& Comp

Eng, Manuf& Const

Ag &Vet

Health &Welfare

Services


Academic Challenge ISSE

All Students

Undergraduate Year 1

Undergraduate Final Yr

Postgraduate

Worked harder than you thought you could to meet a teachers standards or expectations

Never 14.7 20.2 10.4 10.2

Sometimes 39.2 41.9 38.3 34.9

Often 33.1 28.2 36.6 38.0

Very Often 12.9 9.7 14.7 16.9

Analysing the basic elements of an idea, problem, experience or theory, such as examining a particular case or situation in depth and considering its components

Very Little 4.2 5.3 3.9 2.3

Some 21.2 25.0 19.5 15.7

Quite a bit 42.8 41.9 45.2 40.7

Very much 31.8 27.8 31.4 41.3

Organising and synthesising ideas, information or experiences into new, more complex interpretations and relationships

Very Little 10.7 13.8 9.8 5.0

Some 28.5 32.8 28.2 19.7

Quite a bit 38.5 35.5 40.8 40.9

Very much 22.3 17.8 21.2 34.4

Making judgements about the value of information, arguments or methods (e.g. examining how others gather and interpret data and assessing the soundness of their conclusions)

Very Little 11.8 15.6 10.4 5.8

Some 29.2 32.5 29.7 21.2

Quite a bit 35.2 33.1 35.6 39.0

Very much 23.8 18.8 24.3 34.1

Applying theories or concepts to practical problems or in new situations

Very Little 8.6 10.4 8.7 4.3

Some 26.1 27.7 27.0 20.7

Quite a bit 34.9 34.1 34.4 37.8

Very much 30.4 27.9 29.8 37.2

Assigned textbooks, books, book-length packs or journal articles of subject readings have you read?

None 7.8 11.6 6.2 2.5

1 to 4 26.2 36.2 20.7 14.2

5 to 10 20.4 23.7 19.4 15.0

11 to 19 15.7 14.8 17.1 15.1

20+ 29.9 13.7 36.6 53.2

Assignments of fewer than 1000 words or equivalent have you completed

None 30.5 20.7 35.2 43.7

1 to 4 38.8 44.0 36.6 31.2

5 to 10 17.7 19.2 17.7 14.3

11 to 19 7.9 9.7 6.9 6.0

20+ 5.1 6.4 3.6 4.7

Assignments of between 1000 and 5000 words or equivalent have you completed

None 12.1 18.6 6.6 7.3

1 to 4 45.8 51.3 42.0 40.6

5 to 10 32.1 24.9 37.8 38.1

11 to 19 8.4 4.4 11.6 11.7

20+ 1.6 0.9 2.0 2.3

Assignments of more than 5,000 words or equivalent have you completed

None 63.2 87.2 44.0 44.3

1 to 4 32.2 10.0 51.0 47.6

5 to 10 3.0 1.3 3.5 6.2

11 to 19 0.8 0.5 0.9 1.3

20+ 0.8 1.0 0.6 0.6

Preparing for class (e.g. studying, reading, writing, doing homework or lab work, analysing data, rehearsing and other academic activities)

None 4.3 4.4 4.9 2.9

1 to 5 31.2 35.9 28.6 25.5

6 to 10 22.4 24.8 21.1 19.4

11 to 15 15.3 15.8 14.5 15.6

16 to 20 10.8 8.8 11.3 14.2

21 to 25 6.2 4.8 7.4 7.5

26 to 30 3.9 2.8 4.0 6.3

30+ 5.8 2.7 8.2 8.6

Spending significant amounts of time studying and on academic work

Very little 3.6 3.5 4.2 2.7

Some 20.4 22.7 18.8 18.0

Quite a bit 46.4 48.6 45.0 44.1

Very much 29.6 25.2 32.0 35.2


Active LearningActive Learning is defined as the students’ efforts to actively construct knowledge

Index scores:


ISSE UG - Year 1

UG - Final Yr


Part-time

RemoteP

ER

CE

NTI

lES

Mean 40.1 37.5 41.9 42.6 39.1 42.0 38.8 40.6 38.6 26.5

5th 14.3 14.3 16.7 19.0 14.3 19.0 14.3 16.7 14.3 8.3

25th 28.6 28.6 28.6 28.6 28.6 28.6 28.6 28.6 28.6 14.3

Median 38.1 38.1 42.9 42.9 38.1 42.9 38.1 38.1 38.1 23.8

75th 52.4 47.6 52.4 52.4 47.6 52.4 47.6 52.4 47.6 33.3

95th 66.7 61.9 71.4 71.4 66.7 71.4 65.5 66.7 66.7 52.4

Index scores:


23 Years and Under

24 years and over


PE

RC

EN

TIlE

S

Mean 39.6 40.6 42.6 40.0 39.9 42.3 39.4 40.7

5th 14.3 14.3 21.2 14.3 14.3 14.3 14.3 14.3

25th 28.6 28.6 33.3 28.6 28.6 32.5 28.6 28.6

Median 38.1 38.1 42.0 38.1 38.1 42.9 38.1 38.1

75th 47.6 52.4 52.4 52.4 50.0 52.4 47.6 52.4

95th 66.7 66.7 72.2 66.7 66.7 71.4 66.7 66.7

0

20

40

60

80

100

ISSE UGYear 1

UGFinal Year


0

20

40

60

80

100

23 Years & Under

24 Years & Over


0

20

40

60

80

100

95th

75th

Median

25th

5th



Active LearningIndex scores:

By Field of Study


Arts & Humanities


Sc, Maths & Comp

Eng, Manuf &

Const


Services

PE

RC

EN

TIlE

S

Mean 41.8 38.8 41.3 37.2 40.0 39.9 42.5 44.6

5th 19.0 14.3 19.0 14.3 16.7 14.8 19.0 19.0

25th 28.6 28.6 28.6 27.8 28.6 28.6 28.6 33.3

Median 42.9 38.1 38.9 38.1 38.1 38.1 42.9 42.9

75th 52.4 47.6 52.4 47.6 50.0 52.4 52.4 57.1

95th 66.7 66.7 71.4 61.9 66.7 69.2 71.4 76.2

ISSE

All Students



Postgraduate

Asked questions or contributed to discussions in class, tutorials, labs or online

Never 6.2 7.5 6.3 2.9

Sometimes 39.0 45.5 37.1 27.6

Often 30.0 28.7 29.8 33.5

Very Often 24.8 18.4 26.8 36.0

Made a class or online presentation Never 20.4 27.0 13.8 17.4

Sometimes 43.3 46.1 42.7 37.9

Often 24.0 20.5 27.5 25.6

Very Often 12.3 6.4 16.0 19.1

Worked with other students inside class to prepare assignments

Never 16.7 15.8 17.0 18.2

Sometimes 35.8 37.5 34.4 34.7

Often 31.9 32.9 31.4 30.2

Very Often 15.6 13.8 17.2 16.9

Worked with other students outside class to prepare assignments

Never 21.3 22.1 19.7 22.6

Sometimes 33.7 35.0 32.5 33.2

Often 28.6 29.5 29.0 25.7

Very Often 16.4 13.4 18.8 18.5

Tutored or taught other college students (paid or voluntary)

Never 70.3 72.5 66.9 71.7

Sometimes 20.7 19.6 23.5 18.2

Often 6.4 6.1 6.5 6.8

Very Often 2.6 1.8 3.1 3.3

Participated in a community-based project (e.g. volunteering) as part of your course

Never 80.4 82.5 76.7 82.8

Sometimes 11.3 10.3 13.7 9.2

Often 4.9 4.2 6.0 4.3

Very Often 3.4 3.1 3.7 3.7

Discussed ideas from your coursework with others outside class (e.g. students, family members, co-workers, etc)

Never 6.1 6.9 6.5 3.8

Sometimes 32.2 32.6 31.8 31.8

Often 36.2 35.7 36.9 36.1

Very Often 25.5 24.8 24.9 28.2

0

20

40

60

80

100

Education& Training

Arts &Humanities


Sci, Maths& Comp

Eng, Manuf& Const

Ag &Vet

Health &Welfare

Services

0

20

40

60

80

100

95th

75th

Median

25th

5th



Student Staff InteractionsStudent Staff Interactions is defined as the level and nature of students’ contact and interactions with teaching staff

Index scores:


ISSE UG - Year 1

UG - Final Yr


Part-time

Remote

PE

RC

EN

TIlE

S

Mean 22.1 17.9 25.5 25.3 21.7 22.9 21.1 22.2 21.8 19.4

5th 0.0 0.0 5.6 5.6 0.0 0.0 0.0 0.0 4.2 5.6

25th 11.1 5.6 11.1 11.1 11.1 11.1 11.1 11.1 11.1 11.1

Median 16.7 16.7 22.2 22.2 16.7 22.2 16.7 16.7 16.7 16.7

75th 27.8 22.2 33.3 33.3 27.8 33.3 27.8 33.3 27.8 26.9

95th 55.6 46.7 61.1 55.6 55.6 55.6 50.0 55.6 50.0 38.9

Index scores:


23 Years and Under

24 years and over


PE

RC

EN

TIlE

S

Mean 20.7 24.0 21.9 22.0 24.5 20.3 22.5 21.7

5th 0.0 5.6 0.0 0.0 2.7 0.0 0.0 0.0

25th 11.1 11.1 11.1 11.1 11.1 11.1 11.1 11.1

Median 16.7 22.2 22.2 16.7 22.2 16.7 16.7 16.7

75th 27.8 33.3 33.3 27.8 33.3 27.8 33.3 27.8

95th 55.6 55.6 48.1 55.6 55.6 52.7 55.6 55.6

0

20

40

60

80

100

ISSE UGYear 1

UGFinal Year


23 Years & Under

24 Years & Over

0102030405060708090

100

0

20

40

60

80

100



Student Staff InteractionsIndex scores:

By Field of Study


Arts & Humanities


Sc, Maths & Comp

Eng, Manuf &

Const


Services

PE

RC

EN

TIlE

S

Mean 22.0 23.6 22.0 20.6 21.0 19.8 22.2 25.8

5th 0.0 5.6 0.0 0.0 0.0 0.0 0.0 1.8

25th 11.1 11.1 11.1 11.1 11.1 5.6 11.1 11.1

Median 16.7 22.2 16.7 16.7 16.7 16.7 16.7 22.2

75th 27.8 33.3 30.3 27.8 27.8 27.8 33.3 33.3

95th 50.0 50.0 55.6 50.9 50.0 55.6 55.6 61.1

ISSE

All Students



Postgraduate

Discussed your grades or assignments with teaching staff/tutors

Never 32.3 39.0 26.3 28.3

Sometimes 44.5 43.0 46.0 45.0

Often 17.2 13.6 20.5 19.3

Very Often 6.0 4.5 7.2 7.4

Talked about your career plans with teaching staff or career advisors

Never 56.9 70.2 44.1 50.3

Sometimes 30.7 23.3 38.1 34.0

Often 9.3 5.1 13.1 12.0

Very Often 3.1 1.5 4.7 3.8

Discussed ideas from your course-work or classes with teaching staff outside class

Never 49.5 62.7 41.6 35.1

Sometimes 36.2 28.5 40.5 45.2

Often 11.4 7.0 14.3 15.8

Very Often 2.9 1.9 3.6 3.9

Received timely written or oral feedback from teachers/tutors on your academic performance

Never 18.4 22.3 16.2 13.3

Sometimes 46.8 45.1 48.9 46.6

Often 26.2 24.5 26.6 29.2

Very Often 8.7 8.0 8.3 10.9

Worked with teaching staff on activities other than coursework (e.g. committees, orientation, student organisations etc.)

Never 76.0 80.9 70.9 74.1

Sometimes 16.7 13.6 19.6 18.5

Often 5.3 4.0 6.7 5.5

Very Often 2.0 1.5 2.7 1.9

Work on a research project with a staff member outside of coursework requirements

Do not know about 24.2 29.5 20.7 18.6

Have not decided 21.1 29.4 13.7 15.9

Do not plan to do 36.4 22.4 49.5 44.0

Plan to do 13.6 16.5 8.7 15.9

Done 4.8 2.2 7.5 5.7

0

20

40

60

80

100

Education& Training

Arts &Humanities


Sci, Maths& Comp

Eng, Manuf& Const

Ag &Vet

Health &Welfare

Services

0

20

40

60

80

100

95th

75th

Median

25th

5th



Enriching Educational ExperiencesEnriching Educational Experiences is defined as students’ participation in broadening educational activities

Index scores:


ISSE UG - Year 1

UG - Final Yr


Part-time

Remote

PE

RC

EN

TIlE

S

Mean 27.9 26.2 30.4 27.4 29.2 25.7 26.9 29.1 22.4 22.5

5th 8.3 8.3 8.3 7.4 8.3 7.7 7.9 8.3 5.6 2.8

25th 16.7 16.7 17.9 16.7 17.9 16.0 16.1 17.9 12.3 11.1

Median 25.0 25.0 27.8 25.0 26.2 23.4 24.2 26.2 19.4 20.0

75th 36.1 33.3 40.7 36.1 37.3 33.3 34.5 37.3 30.3 30.6

95th 56.7 49.6 60.7 59.5 58.8 51.2 56.7 57.5 45.6 47.2

Index scores:


23 Years and Under

24 years and over


PE

RC

EN

TIlE

S

Mean 29.7 25.6 24.1 27.7 34.4 25.1 26.4 29.4

5th 9.5 6.1 5.6 8.3 11.1 8.3 8.3 8.3

25th 19.0 15.1 13.9 16.7 21.8 16.7 16.5 17.9

Median 27.8 23.4 21.3 25.0 31.6 22.2 24.6 27.4

75th 38.5 33.3 33.8 36.1 44.4 31.7 34.5 38.1

95th 57.9 53.9 58.0 55.6 69.0 49.7 54.0 58.3

0

20

40

60

80

100

ISSE UGYear 1

UGFinal Year


0

20

40

60

80

100

23 Years & Under

24 Years & Over



Enriching Educational ExperiencesIndex scores:

By Field of Study


Arts & Humanities


Sc, Maths & Comp

Eng, Manuf &

Const


Services

PE

RC

EN

TIlE

S

Mean 26.9 28.2 27.8 27.1 25.6 28.6 30.7 28.4

5th 6.5 8.3 8.3 8.3 5.6 9.5 8.3 7.5

25th 16.3 16.7 16.7 16.7 14.8 16.7 20.2 16.7

Median 23.4 26.2 25.0 25.0 23.4 26.2 29.0 27.2

75th 34.5 37.3 35.7 34.6 34.1 36.1 40.1 37.3

95th 57.9 57.3 56.0 54.0 53.0 56.7 57.9 58.2

0

20

40

60

80

100

Education& Training

Arts &Humanities


Sci, Maths& Comp

Eng, Manuf& Const

Ag &Vet

Health &Welfare

Services

0

20

40

60

80

100

95th

75th

Median

25th

5th



Enriching Educational Experiences

ISSE

All Students



Postgraduate

Used an online learning system to discuss or complete an assignment e.g. Moodle, Blackboard

Never 16.8 16.2 17.1 17.6

Sometimes 22.9 21.1 24.1 24.4

Often 23.8 23.8 24.2 23.3

Very Often 36.5 38.9 34.7 34.7

Had conversations with students of a different ethnicity/nationality than your own

Never 10.6 9.8 10.9 11.7

Sometimes 31.5 29.1 34.2 32.2

Often 30.0 31.0 29.1 29.6

Very Often 27.8 30.1 25.8 26.6

Had conversations with students who are very different to you in terms of their religious beliefs, political opinions or personal values

Never 11.6 11.3 11.7 12.3

Sometimes 36.4 33.9 37.6 39.8

Often 28.9 29.2 28.8 28.3

Very Often 23.1 25.6 21.9 19.6

Encouraging contact among students from different economic, social and ethnic/national backgrounds

Very Little 33.2 29.5 36.2 36.0

Some 33.5 33.4 34.5 31.7

Quite a bit 23.0 24.7 21.4 22.0

Very much 10.4 12.4 7.9 10.4

Community service or volunteer work

Do not know about 8.9 8.7 9.0 9.3

Have not decided 20.0 23.4 18.2 15.3

Do not plan to do 23.2 12.8 28.9 36.3

Plan to do 24.4 36.0 14.8 15.9

Done 23.5 19.2 29.0 23.3

Internship, fieldwork or clinical placement

Do not know about 15.7 18.9 13.6 12.4

Have not decided 17.5 21.2 16.0 11.9

Do not plan to do 25.1 13.8 30.9 40.0

Plan to do 28.1 41.1 17.1 18.8

Done 13.5 4.9 22.4 16.9

Participate in a study group or learning community

Do not know about 15.0 15.5 15.0 13.9

Have not decided 19.6 26.2 14.2 14.8

Do not plan to do 28.2 16.1 38.1 37.5

Plan to do 16.5 23.8 9.1 13.7

Done 20.7 18.5 23.5 20.2

Study a foreign language Do not know about 8.5 8.1 8.8 8.8

Have not decided 14.4 17.8 12.1 11.2

Do not plan to do 40.7 33.6 44.4 50.1

Plan to do 17.7 19.3 16.7 16.1

Done 18.6 21.2 18.0 13.9

Study abroad or student exchange Do not know about 10.2 9.7 10.3 10.9

Have not decided 17.2 24.8 12.2 9.8

Do not plan to do 47.2 31.6 57.1 63.5

Plan to do 18.0 31.3 8.8 5.8

Done 7.3 2.6 11.6 9.9

Culminating final-year experience (e.g. honours thesis,final year project, comprehensive exam, etc)


Have not decided 13.9 21.0 9.2 7.1

Do not plan to do 9.9 6.1 12.7 13.1

Plan to do 50.5 49.0 47.4 60.3

Done 11.2 1.0 22.9 11.6

Independent study (e.g. outside your course)


Have not decided 20.8 27.5 16.0 14.5

Do not plan to do 21.8 13.6 28.3 28.5

Plan to do 31.2 33.1 28.7 31.3

Done 17.0 13.3 20.4 19.1

Participating in extracurricular ac-tivities (e.g. organisations, campus publications, student associations, clubs and societies, sports, etc.)

None 46.1 41.3 47.6 54.2

1 to 5 32.0 35.4 30.0 28.1

6 to 10 12.8 13.7 12.7 10.7

11 to 15 5.2 5.7 5.3 3.8

16 to 20 2.0 2.1 1.9 1.8

21 to 25 1.0 1.0 1.2 0.6

26 to 30 0.4 0.3 0.6 0.4

30+ 0.5 0.5 0.7 0.3


Supportive Learning EnvironmentSupportive Learning Environment is defined as students’ feelings of support within the university community

Index scores:


ISSE UG - Year 1

UG - Final Yr


Part-time

Remote

PE

RC

EN

TIlE

S

Mean 54.0 55.6 52.3 53.6 54.3 53.5 54.4 54.5 52.0 50.7

5th 25.0 25.0 25.0 25.0 25.0 25.0 25.0 25.0 25.0 22.0

25th 41.7 44.4 40.0 41.7 41.7 41.7 41.7 41.7 40.0 40.0

Median 53.3 55.6 52.8 52.8 53.3 52.8 53.3 54.2 52.8 47.2

75th 66.7 66.7 63.9 63.9 66.7 63.9 66.7 66.7 62.5 61.1

95th 83.3 86.1 83.3 83.3 83.3 83.3 87.7 83.3 80.6 83.3

Index scores:


23 Years and Under

24 years and over


PE

RC

EN

TIlE

S

Mean 54.5 53.3 59.8 54.0 56.7 48.3 53.8 54.2

5th 25.0 25.0 32.9 25.0 27.8 19.4 25.0 25.0

25th 41.7 41.7 50.0 41.7 43.3 36.1 41.7 41.7

Median 55.6 52.8 55.6 53.3 58.3 47.2 53.3 53.3

75th 66.7 63.9 72.2 66.7 70.0 58.5 66.7 66.7

95th 86.1 83.3 92.4 83.3 86.1 77.8 83.3 86.1

0

20

40

60

80

100

ISSE UGYear 1

UGFinal Year


0

20

40

60

80

100

23 Years & Under

24 Years & Over


0

20

40

60

80

100

95th

75th

Median

25th

5th



Supportive Learning EnvironmentIndex scores:

By Field of Study


Arts & Humanities


Sc, Maths & Comp

Eng, Manuf &

Const


Services

PE

RC

EN

TIlE

S

Mean 53.5 54.6 53.5 54.4 53.2 54.2 54.5 54.2

5th 23.3 25.0 25.0 25.0 27.8 25.0 25.0 25.0

25th 40.3 41.7 41.7 41.7 41.7 41.7 41.7 41.7

Median 52.8 55.6 52.8 53.3 52.8 52.8 55.6 52.8

75th 66.7 66.7 66.7 66.7 63.9 66.7 66.7 66.7

95th 86.1 83.3 83.3 83.3 86.1 83.3 83.3 88.9

0

20

40

60

80

100

Education& Training

Arts &Humanities


Sci, Maths& Comp

Eng, Manuf& Const

Ag &Vet

Health &Welfare

Services


Supportive Learning Environment

ISSE

All Students



Postgraduate

Relationships with other students

Unfriendly, unsupportive, sense of alienation

1.2 1.3 1.5 0.5

2 2.5 2.6 2.6 2.1

3 4.8 5.2 4.5 4.5

4 10.1 10.4 10.4 8.8

5 18.4 18.4 18.3 18.7

6 29.3 28.6 28.7 31.8

Friendly, supportive, sense of belonging

33.8 33.6 34.1 33.6

Relationships with teaching staff

Unavailable, unhelpful, unsympathetic

1.3 1.5 1.3 0.9

2 3.7 3.6 4.1 3.1

3 7.0 7.7 6.7 5.9

4 15.7 17.3 15.3 12.6

5 25.4 26.5 24.9 23.8

6 27.4 25.8 27.8 30.1

Available, helpful, sympathetic 19.6 17.5 20.0 23.6

Relationships with administrative personnel

Unavailable, inconsiderate, rigid

3.7 3.2 4.9 2.6

2 6.8 5.7 8.6 5.9

3 10.5 10.3 11.1 9.6

4 22.1 24.1 21.7 18.6

5 22.9 24.2 21.9 22.0

6 19.8 19.1 18.5 23.6

Available, considerate, flexible 14.2 13.3 13.3 17.7

Providing the support you need to help you succeed academically

Very little 7.1 5.9 8.6 7.2

Some 30.1 28.5 32.4 29.9

Quite a bit 43.7 44.3 42.7 44.0

Very much 19.1 21.3 16.3 18.8

Helping you cope with your non-academic responsibilities (e.g. work, family, etc.)

Very little 47.7 41.6 52.0 53.7

Some 32.9 36.0 31.1 29.3

Quite a bit 14.1 16.2 12.6 12.3

Very much 5.2 6.2 4.3 4.8

Providing the support you need to socialise

Very little 38.4 31.3 41.1 49.6

Some 34.8 34.9 35.8 32.6

Quite a bit 19.9 24.4 17.5 13.9

Very much 7.0 9.4 5.6 4.0


Work Integrated LearningWork Integrated Learning is defined as the integration of employment-focused work experiences into study

Index scores:


ISSE UG - Year 1

UG - Final Yr


Part-time

Remote

PE

RC

EN

TIlE

S

Mean 42.9 35.4 47.4 51.7 41.5 43.4 63.7 40.9 52.3 45.5

5th 8.3 0.0 8.3 8.3 8.3 8.3 16.7 8.3 11.1 8.3

25th 25.0 16.7 25.0 33.3 22.2 25.0 44.4 25.0 33.3 25.0

Median 41.7 33.3 50.0 50.0 33.3 41.7 66.7 33.3 50.0 41.7

75th 58.3 50.0 66.7 66.7 58.3 58.3 83.3 58.3 66.7 66.7

95th 91.7 83.3 91.7 100.0 91.7 91.7 100.0 91.7 100.0 100.0

Index scores:

By Age cohort By Domicile Group By Sex

23 Years and Under

24 years and over


PE

RC

EN

TIlE

S

Mean 38.6 48.2 66.1 43.1 40.6 39.7 40.4 45.4

5th 8.3 8.3 25.0 8.3 8.3 8.3 8.3 8.3

25th 16.7 25.0 50.0 25.0 25.0 25.0 25.0 25.0

Median 33.3 50.0 66.7 41.7 33.3 33.3 33.3 41.7

75th 58.3 66.7 83.3 58.3 58.3 58.3 58.3 66.7

95th 91.7 91.7 100.0 91.7 83.3 91.7 83.3 91.7

0

20

40

60

80

100

ISSE UGYear 1

UGFinal Year


0

20

40

60

80

100

23 Years & Under

24 Years & Over


0

20

40

60

80

100

95th

75th

Median

25th

5th



Work Integrated LearningIndex scores:

By Field of Study


Arts & Humanities


Sc, Maths & Comp

Eng, Manuf &

Const


Services

PE

RC

EN

TIlE

S

Mean 67.3 32.6 41.1 36.0 40.6 52.2 58.6 51.8

5th 25.0 0.0 8.3 8.3 8.3 8.3 16.7 8.3

25th 50.0 16.7 25.0 16.7 25.0 33.3 41.7 33.3

Median 66.7 33.3 41.7 33.3 41.7 58.3 58.3 50.0

75th 83.3 41.7 58.3 50.0 58.3 75.0 83.3 71.6

95th 100.0 75.0 83.3 83.3 83.3 98.5 100.0 94.5

ISSE

All Students



Postgraduate

Blended academic learning with workplace experience

Never 37.4 49.0 31.6 21.9

Sometimes 26.9 26.4 29.3 23.9

Often 20.3 15.6 23.1 26.0

Very Often 15.4 9.1 16.1 28.2

Improved knowledge and skills that will contribute to your employability

Never 7.5 9.3 7.2 3.9

Sometimes 30.2 32.1 31.7 23.3

Often 39.9 39.3 39.3 42.0

Very Often 22.4 19.3 21.7 30.8

Explored how to apply your learning in the workplace

Never 20.4 25.1 19.2 12.0

Sometimes 32.2 33.8 32.4 28.0

Often 29.8 27.0 30.6 34.8

Very Often 17.6 14.1 17.7 25.1

Industry placement or work experience

Do not know about 10.4 10.0 10.4 11.2

Have not decided 13.4 14.8 12.9 10.8

Do not plan to do 16.6 5.9 21.5 32.0

Plan to do 33.7 55.9 14.9 17.9

Done 25.9 13.3 40.4 28.1

0

20

40

60

80

100

Education& Training

Arts &Humanities


Sci, Maths& Comp

Eng, Manuf& Const

Ag &Vet

Health &Welfare

Services

0

20

40

60

80

100

95th

75th

Median

25th

5th



Higher Order ThinkingHigher Order Thinking is defined as participation in higher order forms of thinking

Index scores:


ISSE UG - Year 1

UG - Final Yr


Part-time

RemoteP

ER

CE

NTI

lES

Mean 61.1 57.1 61.4 69.6 63.7 56.2 61.6 60.9 62.3 57.8

5th 16.7 16.7 25.0 33.3 25.0 16.7 25.0 16.7 16.7 8.3

25th 41.7 41.7 44.4 58.3 50.0 41.7 44.4 41.7 41.7 41.7

Median 66.7 58.3 66.7 66.7 66.7 58.3 66.7 58.3 66.7 58.3

75th 75.0 75.0 75.0 88.9 83.3 75.0 75.0 75.0 83.3 75.0

95th 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0

Index scores:


23 Years and Under

24 years and over


PE

RC

EN

TIlE

S

Mean 60.5 61.9 61.1 61.1 64.2 54.9 59.7 62.5

5th 16.7 16.7 25.5 16.7 25.0 16.7 16.7 16.7

25th 41.7 41.7 44.4 41.7 50.0 33.3 41.7 44.4

Median 58.3 66.7 60.4 66.7 66.7 58.3 58.3 66.7

75th 75.0 83.3 75.0 75.0 83.3 66.7 75.0 83.3

95th 100.0 100.0 93.7 100.0 100.0 91.7 100.0 100.0

0

20

40

60

80

100

ISSE UGYear 1

UGFinal Year


0

20

40

60

80

100

23 Years & Under

24 Years & Over



Higher Order Thinking

Index scores:

By Field of Study


Arts & Humanities


Sc, Maths & Comp

Eng, Manuf &

Const


ServicesP

ER

CE

NTI

lES

Mean 66.4 60.9 62.9 59.2 58.7 57.1 63.7 53.2

5th 25.0 16.7 25.0 16.7 16.7 16.7 25.0 16.7

25th 50.0 50.0 50.0 41.7 41.7 41.7 50.0 33.3

Median 66.7 66.7 66.7 58.3 58.3 58.3 66.7 50.0

75th 83.3 75.0 83.3 75.0 75.0 75.0 83.3 66.7

95th 100.0 100.0 100.0 100.0 100.0 91.7 100.0 100.0

ISSE

All Students



Postgraduate


Very Little 10.7 13.8 9.8 5.0

Some 28.5 32.8 28.2 19.7

Quite a bit 38.5 35.5 40.8 40.9

Very much 22.3 17.8 21.2 34.4

Making judgements about the value of information, arguments or methods (e.g. examining how others gather and interpret data and assessing the soundness of their conclusions)

Very Little 11.8 15.6 10.4 5.8

Some 29.2 32.5 29.7 21.2

Quite a bit 35.2 33.1 35.6 39.0

Very much 23.8 18.8 24.3 34.1


Very Little 8.6 10.4 8.7 4.3

Some 26.1 27.7 27.0 20.7

Quite a bit 34.9 34.1 34.4 37.8

Very much 30.4 27.9 29.8 37.2

Analysing the basic elements of an idea,problem, experience or theory, such as examining a particular case or situation in depth and considering its components

Very Little 4.2 5.3 3.9 2.3

Some 21.2 25.0 19.5 15.7

Quite a bit 42.8 41.9 45.2 40.7

Very much 31.8 27.8 31.4 41.3

0

20

40

60

80

100

Education& Training

Arts &Humanities


Sci, Maths& Comp

Eng, Manuf& Const

Ag &Vet

Health &Welfare

Services

0

20

40

60

80

100

95th

75th

Median

25th

5th



General Learning OutcomesGeneral Learning Outcomes is defined as the development of general competencies

Index scores:


ISSE UG - Year 1

UG - Final Yr


Part-time

RemoteP

ER

CE

NTI

lES

Mean 61.5 58.3 64.5 62.7 61.6 61.3 61.1 62.0 59.0 61.0

5th 25.0 20.8 25.0 25.0 25.0 20.8 25.1 25.0 20.8 22.2

25th 45.8 45.8 50.0 45.8 45.8 45.8 45.8 45.8 41.7 42.9

Median 62.5 58.3 66.7 62.5 62.5 62.5 62.5 62.5 58.3 62.5

75th 79.2 79.2 79.2 79.2 75.0 79.2 79.2 75.0 75.0 79.2

95th 95.8 91.7 100.0 100.0 95.8 95.8 91.7 95.8 95.8 95.8

Index scores:


23 Years and Under

24 years and over


PE

RC

EN

TIlE

S

Mean 61.5 61.4 60.6 61.6 59.6 59.3 60.8 62.1

5th 25.0 24.3 15.9 25.0 25.0 16.7 20.8 25.0

25th 45.8 45.8 41.5 45.8 45.8 45.8 45.8 45.8

Median 62.5 62.5 62.5 62.5 58.3 62.5 62.5 62.5

75th 77.8 79.2 82.8 79.2 75.0 75.0 77.8 79.2

95th 95.8 100.0 96.8 95.8 95.8 95.8 95.8 100.0

0

20

40

60

80

100

ISSE UGYear 1

UGFinal Year


0

20

40

60

80

100

23 Years & Under

24 Years & Over



General Learning OutcomesIndex scores:

By Field of Study


Arts & Humanities


Sc, Maths & Comp

Eng, Manuf &

Const


Services

PE

RC

EN

TIlE

S

Mean 61.9 59.2 62.1 60.9 62.4 57.4 63.8 61.2

5th 25.0 20.8 23.8 24.6 25.0 16.7 25.0 20.8

25th 45.8 45.8 45.8 45.8 50.0 41.7 47.6 41.7

Median 62.5 58.3 62.5 62.5 62.5 62.5 66.7 62.5

75th 79.2 75.0 79.2 75.0 77.8 75.0 79.2 79.2

95th 100.0 100.0 100.0 95.8 95.8 95.8 100.0 100.0

0

20

40

60

80

100

Education& Training

Arts &Humanities


Sci, Maths& Comp

Eng, Manuf& Const

Ag &Vet

Health &Welfare

Services

0

20

40

60

80

100

95th

75th

Median

25th

5th



General Learning Outcomes

ISSE

All Students



Postgraduate

Acquiring job-related or work-related knowledge and skills

Very Little 12.4 15.6 10.8 8.6

Some 27.5 31.5 24.9 23.3

Quite a bit 33.3 31.9 33.9 35.2

Very much 26.8 21.0 30.4 32.9

Writing clearly and effectively Very Little 11.4 14.6 8.5 9.5

Some 26.1 30.5 22.9 21.9

Quite a bit 36.7 35.0 37.4 39.2

Very much 25.8 19.8 31.2 29.4

Speaking clearly and effectively Very Little 14.2 16.9 11.1 14.4

Some 29.2 31.9 27.1 27.2

Quite a bit 34.4 32.8 36.1 34.6

Very much 22.2 18.4 25.7 23.8

Thinking critically and analytically Very Little 3.7 4.7 3.2 2.8

Some 18.4 20.8 16.7 16.0

Quite a bit 39.0 40.4 36.9 39.8

Very much 38.9 34.1 43.1 41.4

Analysing quantitative problems Very Little 10.4 11.8 8.6 10.6

Some 26.6 28.6 25.7 23.9

Quite a bit 36.1 36.3 35.5 36.9

Very much 26.9 23.3 30.3 28.6

Using computing and information technology

Very Little 9.4 9.6 7.8 12.2

Some 23.3 23.6 21.6 25.7

Quite a bit 32.8 33.4 33.4 30.5

Very much 34.5 33.4 37.2 31.6

Working effectively with others Very Little 7.4 7.3 7.0 8.4

Some 26.2 26.3 24.9 28.2

Quite a bit 38.6 40.3 37.9 36.1

Very much 27.9 26.1 30.2 27.3

Learning effectively on your own Very Little 8.2 9.0 6.9 9.0

Some 25.2 28.4 22.4 23.5

Quite a bit 38.1 39.0 38.1 36.0

Very much 28.5 23.7 32.5 31.5


General Development OutcomesGeneral Development Outcomes is defined as the formation of general forms of individual and social development

Index scores:


ISSE UG - Year 1

UG - Final Yr


Part-time

Remote

PE

RC

EN

TIlE

S

Mean 41.5 41.2 42.4 40.6 42.6 39.4 43.2 42.8 36.2 31.5

5th 5.6 5.6 0.0 0.0 5.6 0.0 7.4 5.6 0.0 0.0

25th 22.2 22.2 22.2 22.2 22.2 16.7 27.8 22.2 16.7 11.1

Median 38.9 38.9 38.9 38.9 38.9 38.9 38.9 38.9 33.3 27.8

75th 61.1 60.9 61.1 61.1 61.1 55.6 56.1 61.1 55.6 44.4

95th 86.7 83.3 88.9 86.4 88.9 83.3 80.0 88.9 83.3 95.9

Index scores:


23 Years and Under

24 years and over


PE

RC

EN

TIlE

S

Mean 43.2 39.4 40.6 41.5 44.5 36.9 39.1 43.8

5th 5.6 0.0 0.0 5.6 5.6 5.6 0.0 5.6

25th 22.2 16.7 27.5 22.2 27.8 16.7 22.2 22.2

Median 41.7 38.9 38.9 38.9 44.4 33.3 38.9 44.4

75th 61.1 55.6 59.9 61.1 61.1 50.0 55.6 61.1

95th 86.7 86.7 79.3 86.7 88.9 79.9 83.3 88.9

0

20

40

60

80

100

ISSE UGYear 1

UGFinal Year


0

20

40

60

80

100

23 Years & Under

24 Years & Over


0

20

40

60

80

100

95th

75th

Median

25th

5th



General Development OutcomesIndex scores:

By Field of Study


Arts & Humanities


Sc, Maths & Comp

Eng, Manuf &

Const


Services

PE

RC

EN

TIlE

S

Mean 45.5 40.4 42.4 35.7 37.7 35.5 50.4 38.5

5th 11.1 0.0 5.6 0.0 0.0 0.0 5.6 0.0

25th 27.8 22.2 22.2 16.7 16.7 16.7 30.0 16.7

Median 44.4 38.9 38.9 33.3 38.9 33.3 50.0 33.3

75th 61.1 55.6 61.1 50.0 55.6 55.6 72.2 55.6

95th 91.2 83.3 83.3 77.8 77.8 72.2 94.4 89.6

ISSE

All Students



Postgraduate

Voting in local, or national elections or referenda

Very Little 51.9 45.7 51.3 67.2

Some 26.3 29.0 26.9 18.8

Quite a bit 14.3 16.7 14.2 9.0

Very much 7.6 8.6 7.6 5.0

Understanding yourself (e.g. self reflection)

Very Little 20.0 21.4 19.0 18.8

Some 28.5 30.6 27.0 26.4

Quite a bit 29.9 29.4 30.9 29.2

Very much 21.6 18.6 23.0 25.6

Understanding people of other racial, ethnic and national backgrounds

Very Little 28.0 27.2 27.3 31.3

Some 33.1 34.0 32.6 32.3

Quite a bit 24.9 25.9 24.5 23.1

Very much 14.0 12.9 15.6 13.4

Solving complex, real-world problems

Very Little 17.8 19.4 17.2 15.2

Some 32.3 33.4 32.3 29.9

Quite a bit 31.8 30.6 31.9 34.2

Very much 18.2 16.7 18.7 20.7

Developing a personal code of values and ethics

Very Little 24.6 25.6 22.5 26.1

Some 31.1 32.6 30.6 28.7

Quite a bit 27.6 27.1 28.5 26.8

Very much 16.8 14.6 18.4 18.4

Contributing to the welfare of your community

Very Little 35.6 32.7 36.3 40.9

Some 33.9 36.1 33.4 30.0

Quite a bit 20.7 21.8 20.5 19.0

Very much 9.7 9.4 9.8 10.2

0

20

40

60

80

100

Education& Training

Arts &Humanities


Sci, Maths& Comp

Eng, Manuf& Const

Ag &Vet

Health &Welfare

Services


Career ReadinessCareer Readiness is defined as the preparation for participation in the professional workforce

Index scores:


ISSE UG - Year 1

UG - Final Yr


Part-time

Remote

PE

RC

EN

TIlE

S

Mean 40.3 35.1 45.4 42.6 39.5 42.1 36.7 40.8 38.0 41.1

5th 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

25th 20.0 13.3 26.7 20.0 20.0 20.0 13.3 20.0 20.0 13.3

Median 40.0 33.3 46.7 40.0 33.3 40.0 33.3 40.0 33.3 33.3

75th 60.0 53.3 66.7 60.0 60.0 60.0 53.3 60.0 53.3 66.7

95th 91.7 86.7 93.3 93.3 91.7 93.3 86.7 91.7 86.7 100.0

Index scores:


23 Years and Under

24 years and over


PE

RC

EN

TIlE

S

Mean 41.2 39.2 37.2 40.2 42.1 38.2 38.9 41.6

5th 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

25th 20.0 20.0 20.0 20.0 20.0 20.0 20.0 20.0

Median 40.0 33.3 40.0 40.0 40.0 33.3 33.3 40.0

75th 60.0 60.0 53.3 60.0 60.0 53.3 60.0 60.0

95th 93.3 86.7 80.0 93.3 86.7 92.7 86.7 93.3

0

20

40

60

80

100

ISSE UGYear 1

UGFinal Year


0

20

40

60

80

100

23 Years & Under

24 Years & Over


0

20

40

60

80

100

95th

75th

Median

25th

5th



Career ReadinessIndex scores:

By Field of Study


Arts & Humanities


Sc, Maths & Comp

Eng, Manuf &

Const


Services

PE

RC

EN

TIlE

S

Mean 39.8 35.9 43.5 38.8 39.9 40.3 39.5 48.6

5th 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

25th 20.0 13.3 20.0 20.0 20.0 20.0 20.0 26.7

Median 33.3 33.3 40.0 33.3 33.3 40.0 40.0 46.7

75th 60.0 53.3 60.0 60.0 60.0 60.0 58.3 66.7

95th 93.3 86.7 93.3 86.7 86.7 86.7 86.7 100.0

ISSE

All Students



Postgraduate

Spent time keeping your CV up-to-date

Never 38.4 44.2 32.8 36.1

Sometimes 34.9 32.7 36.9 35.9

Often 17.4 15.5 19.6 17.5

Very Often 9.4 7.6 10.8 10.6

Thought about how to present yourself to potential employers

Never 19.5 24.8 14.6 16.8

Sometimes 36.2 37.7 34.4 36.3

Often 28.6 26.0 31.8 28.8

Very Often 15.6 11.6 19.3 18.1

Explored where to look for jobs relevant to your interests

Never 20.3 26.2 13.8 18.5

Sometimes 36.0 38.1 34.1 34.7

Often 26.8 23.7 29.9 28.1

Very Often 16.9 12.0 22.1 18.6

Used networking to source information on job opportunities

Never 34.6 41.9 27.5 31.2

Sometimes 33.6 31.9 34.0 36.5

Often 20.1 17.1 23.7 20.4

Very Often 11.7 9.1 14.8 11.9

Set career development goals and plans

Never 29.9 37.2 24.6 23.4

Sometimes 35.3 34.4 35.2 37.6

Often 22.1 18.5 25.0 25.1

Very Often 12.6 9.9 15.3 13.9

0

20

40

60

80

100

Education& Training

Arts &Humanities


Sci, Maths& Comp

Eng, Manuf& Const

Ag &Vet

Health &Welfare

Services


Overall SatisfactionOverall Satisfaction is defined as students’ overall satisfaction with their educational experience

Index scores:


ISSE UG - Year 1

UG - Final Yr


Part-time

Remote

PE

RC

EN

TIlE

S

Mean 67.0 68.6 64.4 68.2 68.2 64.7 67.2 66.5 69.0 69.6

5th 22.2 33.3 22.2 22.2 22.2 22.2 22.2 22.2 22.2 32.9

25th 55.6 55.6 50.0 55.6 55.6 55.6 55.6 55.6 55.6 55.6

Median 66.7 66.7 66.7 66.7 66.7 66.7 66.7 66.7 66.7 66.7

75th 77.8 77.8 77.8 88.9 88.9 77.8 77.8 77.8 88.9 88.9

95th 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0

Index scores:


23 Years and Under

24 years and over


PE

RC

EN

TIlE

S

Mean 65.6 68.7 74.2 67.4 62.3 58.0 66.4 67.6

5th 22.2 22.2 42.7 22.2 22.2 11.1 22.2 22.2

25th 55.6 55.6 66.7 55.6 44.4 44.4 55.6 55.6

Median 66.7 66.7 77.8 66.7 66.7 66.7 66.7 66.7

75th 77.8 88.9 88.9 77.8 77.8 77.8 77.8 77.8

95th 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0

0

20

40

60

80

100

ISSE UGYear 1

UGFinal Year


0

20

40

60

80

100

23 Years & Under

24 Years & Over


0

20

40

60

80

100

95th

75th

Median

25th

5th



Overall SatisfactionIndex scores:

By Field of Study


Arts & Humanities


Sc, Maths & Comp

Eng, Manuf &

Const


Services

PE

RC

EN

TIlE

S

Mean 68.4 69.4 66.9 66.5 63.5 65.0 67.8 64.8

5th 22.2 33.3 22.2 22.2 11.1 33.3 22.2 22.2

25th 55.6 55.6 55.6 55.6 55.6 55.6 55.6 55.6

Median 66.7 66.7 66.7 66.7 66.7 66.7 66.7 66.7

75th 88.9 88.9 77.8 77.8 77.8 77.8 77.8 77.8

95th 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0

ISSE

All Students



Postgraduate

Overall, how would you evaluate the quality of academic advice that you have received?

Poor 6.6 5.9 8.0 5.8

Fair 23.8 23.3 26.2 20.5

Good 51.9 53.7 49.1 52.8

Excellent 17.7 17.2 16.7 20.9

Overall, how would you evaluate your entire educational experience at your institution?

Poor 4.3 3.5 5.5 4.0

Fair 16.6 14.7 18.9 16.7

Good 53.6 56.2 51.0 52.7

Excellent 25.5 25.7 24.7 26.6

If you could start all over again would you go to the same institution?

Definitely No 4.3 2.5 6.9 3.4

Probably No 11.5 9.6 13.9 11.2

Probably Yes 42.4 42.1 41.6 44.5

Definitley Yes 41.8 45.7 37.6 40.9

0

20

40

60

80

100

Education& Training

Arts &Humanities


Sci, Maths& Comp

Eng, Manuf& Const

Ag &Vet

Health &Welfare

Services


SEcTIon 4

NATIONAl-lEVEl ANAlYSIS OF ENgAgEMENT ANd OUTCOMES INdICES

4.1 IntroductionHaving looked in detail at the individual items (Section 3), this section presents an analysis of indices for student Engagement or Outcomes from a variety of perspectives including:

— By year/cohort

— By institution-type

— By mode of study

— By programme-type

— By field of study

As outlined earlier, it is important not to interpret the index scores in the graphs to follow as percentage results. Scores range from 0 to 100 but do not signify proportions of the student body that are, for instance, ‘academically engaged’. Responses to individual questions were recorded as Likert scales and were aggregated and transformed into a composite index ranging from 0 to 100, as outlined in section 3.3.

4.2 Year/cohortFigure 4.2.1 presents the scores for engagement indices by year/cohort i.e. students in first year, in their final year or following postgraduate taught programmes. The main observations are that the score for Academic Challenge increases as stage of study progresses from first year undergraduate to postgraduate higher education. This is also the case with the index for Work Integrated Learning. The scores show an apparent contradiction between the trends in Student Staff Interactions and Supportive Learning Environment across the various stages of study. This is worthy of further study but a review of the individual questions suggests that academic staff are less available to first-year students and the higher score in Supportive Learning Environment arises because first years report higher usage and interaction with the broader administrative and academic supports available on campus.


Figure 4.2.1 Engagement – Year/Cohort (Overall)

Figure 4.2.2 presents the indices for outcomes by year/cohort. The chart demonstrates that Higher Order Thinking increases steadily from first year undergraduate to postgraduate levels, as one might expect. The score for the index Overall Satisfaction is marginally lower for final year undergraduate students. This may reflect the academic workload and pressure experienced by final year undergraduate students as they approach the final semester and final examinations.

Figure 4.2.2 Outcomes – Year/Cohort (Overall)

Academic Challenge

ActiveLearning

StudentStaff

Interactions


Supportive Learning

Environment

Work Integrated Learning

First Year

Final Year

Postgraduate Taught

01020304050607080

0

10

20

30

40

50

60

70

80

43.8 50

.2 54.9

37.5 41

.9

42.6

17.9

25.5

25.3

26.2 30

.4

27.4

55.6

52.3

53.6

35.4

47.4 51

.7

First Year

Final Year

Postgraduate Taught

Higher OrderThinking

General LearningOutcomes

General Development

Outcomes

CareerReadiness

OverallSatisfaction

1020304050607080

0

10

20

30

40

50

60

70

80

57.1 61

.4

69.6

58.3

64.5

62.7

41.2

42.4

40.6

35.1

45.4

42.6

68.6

64.4 68

.2

01020304050607080


4.3 Institution-TypeThis section examines the national engagement and outcomes indices by institution-type. The institution-types are: Universities, Institutes of Technology and Colleges of Education. The results are presented for the full cohort of students. However, detailed index scores by year/cohort for the individual institution-types are available in Appendix 7.


Engagement index scores by institution-type, outlined in Figure 4.3.1, are broadly in line across all institution-types. However, there are notably higher scores for Colleges of Education on the Work Integrated Learning index. This may be reflective of the nature of programmes offered by these institutions, where a significant work placement element is part of the course. Indeed, the timing of the national pilot survey coincided, in many Colleges of Education, with this work-placement component. Institutes of Technology show marginally higher scores for Student-Staff Interactions and Active Learning while Universities demonstrate marginally higher scores for Academic Challenge and Enriching Student Experiences.

Scores, by individual institution-type in Appendix 7, show that Academic Challenge, Active Learning and Work Integrated Learning scores increase between cohorts as the stage of study increases (from first year to final year to postgraduate). These scores also show that Student Staff Interactions are lowest among first year undergraduates across all institution-types. This finding raises questions of interest in the context of the first-year experience and of institutions’ ongoing efforts to promote successful progression and completion of higher education programmes.

Figure 4.3.1 Engagement - Institution Type (Overall)

Academic Challenge

ActiveLearning

StudentStaff

Interactions


Supportive Learning

Environment


0

10

20

30

40

50

60

70

80

48.3

39.1 42

.038

.8

40.1

21.7

22.9

21.1

22.1

29.2

25.7

26.9

27.9

54.3

53.5

54.4

54.0

41.5 43

.4

63.7

42.9

49.9

45.2 48

.8

Universities

Institutes of Technology

Colleges of Education

All Institutions

01020304050607080


Figure 4.3.2 shows that the scores for General Learning Outcomes and General Development Outcomes are comparable across all institution-types. Scores, in Appendix 7 for individual institution-types, demonstrate that Higher Order Thinking increases as the stage of study increases across all institution-types. The Higher Order Thinking scores are higher in the Universities than in the Institutes of Technology for all three cohorts (first year, final year, taught postgraduate). Career Readiness scores are marginally higher in the Institutes of Technology than in Universities for all three cohorts.

Figure 4.3.2 Outcomes - Institution Type (Overall)

4.4 Mode of study This section outlines engagement and outcome index scores by mode of study for all institution-types and all cohorts. There are three modes of study outlined: full-time, part-time and remote.

Figure 4.4.1 presents scores for each engagement index by mode of study. The chart demonstrates that scores for Academic Challenge are very similar for each mode of study. Scores for remote learners are lower for Active Learning and Student Staff Interaction, which is understandable as they study off campus. Enriching Educational Experiences scores are notably higher for full-time students, which may be expected given that these students spend more time on campus and interacting with their fellow students. The Work Integrated Learning scores are higher for part-time students, which may demonstrate the fact that part-time students often combine work and study and tend to select programmes of direct relevance to their career development.

Universities


Colleges of Education

All Institutions

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

61.163

.7

56.2 61

.6

61.5

61.6

61.3

61.1

41.5

42.6

39.4 43

.2

40.3

39.5 42

.1

36.7

67.0

68.2

64.7 67

.2

01020304050607080


Figure 4.4.1 Engagement - Mode of Study (Overall)

Figure 4.4.2 presents scores for outcomes indices by mode of study. The scores are broadly comparable for each student cohort, other than General Development Outcomes which is highest for full-time and lowest for remote students.

Figure 4.4.2 Outcomes - Mode of Study (Overall)

Full-time

Part-time

Remote

Academic Challenge

ActiveLearning

Student Staff

Interactions


Supportive Learning

Environment


0

10

20

30

40

50

60

70

80

01020304050607080

48.2

48.6

47.9

40.6

38.6

26.5

22.2

21.8

19.4

29.1

22.4

22.5

54.5

52.0

50.7

40.9

52.3

45.5

Full-time

Part-time

Remote

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

60.9 62.3

57.8 62

.0

59.0 61

.0

42.8

36.2

31.5

40.8

38.0 41

.1

66.5 69

.0

69.6


4.5 Programme-typeThis section provides scores for engagement and outcomes indices by programme-type (i.e. programmes leading to Higher Certificate, Ordinary Bachelor Degree, Honours Bachelor Degree / Higher Diploma, Masters Degree / Postgraduate Diploma) for all cohorts, modes of study and institution-types.

Figure 4.5.1 presents scores for engagement indices by programme-type. Results for Doctoral students are not included due to the very small sample size. The chart demonstrates that scores for Supportive Learning Environment are consistent for all programme types. The score for Academic Challenge increases from Higher Certificate to Ordinary Bachelor Degree to Honours Bachelor Degree to Masters Degree. This would be expected, given the increase in NFQ award level from Higher Certificate to Masters Degree. The Enriching Educational Experiences scores are highest for students undertaking Honours Bachelor Degree programmes. The Work Integrated Learning scores are highest for students on Masters Degree programmes.

Figure 4.5.1 Engagement - Programme Type (Overall)

Academic Challenge

ActiveLearning

Student Staff

Interactions


SupportiveLearning

Environment


0

10

20

30

40

50

60

70

80

Higher/Advanced Cetificate

Ordinary Bachelor Degree

Honours Bachelor Degree/Higher Diploma

Masters Degree/Postgraduate Diploma

01020304050607080

55.3

39.2

40.5

39.2 43

.2

21.6

22.3

21.2 25

.7

23.4

24.5 28

.9

27.5

55.0

53.4

54.1

53.8

44.1

41.3

40.9

51.5

39.7 42

.6 47.9


Figure 4.5.2 presents scores for outcomes indices by programme-type. The chart demonstrates that scores for Higher Order Thinking increase as the programme-type changes from Higher Certificate to Masters Degree and that scores for General Development Outcomes increase according to undergraduate programme type. The scores for Career Readiness are slightly lower for Honours Bachelor Degree than for Higher Certificate or Ordinary Bachelor Degree. This may reflect the more applied nature of programmes leading to a Higher Certificate and Ordinary Bachelor Degree.

Figure 4.5.2 Outcomes - Programme Type (Overall)

4.6 Field of studyThis section provides the scores for engagement and outcomes indices by field of study for all years of study, modes of study and institution-type.

Figure 4.6.1 presents engagement index scores by field of study. The chart demonstrates that Work Integrated Learning scores are highest in Education & Training and in Health & Welfare disciplines. This is likely to reflect the significant ‘on the job’ aspects of both disciplines. Work Integrated Learning scores are lowest in Arts & Humanities whereas scores for Student Staff Interactions are higher in Arts & Humanities than in other fields of study. The lower scores for Academic Challenge in Engineering, Manufacturing & Construction and in Science, Mathematics and Computing might be regarded as surprising and may be related to the extent to which questions captured the experiences of students pursuing disciplines with significant practical, “non-written” elements. The Supportive Learning Environment scores are similar for all fields of study.

Higher/Advanced Cetificate

Ordinary Bachelor Degree

Honours Bachelor Degree/Higher Diploma

Masters Degree/Postgraduate Diploma

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

70.0

49.6 53

.1

60.8 63.1

59.4

59.2 61

.6

40.7

37.3 38.8 42

.4

42.2

42.6

41.3

39.5

68.4

69.4

65.9

66.7

01020304050607080


Figure 4.6.1 Engagement - Field of Study (Overall)

Education and Training

Arts and Humanities

Social Science, Business & Law

Science, Maths, Computing (inc. Ag & Vet)

Engineering, Manufacturing & Construction

Heath & Welfare

Other & Unknown

0 10 20 30 40 50 60 70 80

0 10 20 30 40 50 60 70 80

AcademicChallenge

Active Learning

Student Staff

Interactions

EnrichingEducational Experiences

SupportiveLearning

Environment


51.749.649.3

45.746.0

49.747.1

41.838.8

41.337.4

40.042.5

39.9

22.023.6

22.020.621.0

22.225.0

26.928.227.8

27.225.6

30.729.2

53.554.6

53.554.4

53.254.554.5

67.332.6

41.137.4

40.658.6

39.9


Figure 4.6.2 presents scores for outcomes indices by field of study. Scores for Higher Order Thinking vary considerably across fields of study. The chart demonstrates that scores for General Learning Outcomes are similar across all fields of study while scores for General Development Outcomes are highest in Health & Welfare and lowest in Science, Mathematics & Computing. The Overall Satisfaction score is broadly similar for most fields of study with highest scores for Arts & Humanities and lowest scores for Engineering, Manufacturing & Construction. Career Readiness scores are highest in Social Science, Business & Law.

Figure 4.6.2 Outcomes - Field of Study (Overall)

Education and Training

Arts and Humanities

Social Science, Business & Law

Science, Maths, Computing (inc. Ag & Vet)

Engineering, Manufacturing & Construction

Heath & Welfare

Other & Unknown

0 10 20 30 40 50 60 70 80

0 10 20 30 40 50 60 70

HigherOrder

Thinking

66.4

GeneralLearning

Outcomes

GeneralDevelopment

Outcomes

CareerReadiness

OverallSatisfaction

60.9

62.9

59.0

58.7

63.755.8

61.9

59.2

62.1

60.5

62.4

63.8

60.9

45.5

40.4

42.4

35.7

37.7

50.4

43.2

39.835.9

43.5

38.9

39.939.5

43.4

68.4

69.4

66.9

66.3

63.5

67.8

66.1


4.7 Student characteristicsThis final section examines scores for engagement and outcomes indices according to the following selected student characteristics:

— Gender

— Age

— Domiciliary

4.7.1. Gender

Figure 4.7.1.1 presents scores for engagement indices by gender for all years of study, all modes of study and all institution-types. The chart demonstrates that the scores for all engagement indices are broadly equivalent for male and female students. However, the scores for Academic Challenge, Enriching Educational Experience and for Work Integrated Learning are somewhat higher for female students.

Figure 4.7.1.1 Engagement - Gender (Overall)

Figure 4.7.1.2 presents the scores for outcomes indices by gender for all years of study, all modes of study and all institution-types. The chart demonstrates that the Outcomes scores are higher in all cases for female students with the greatest differences for General Development Outcomes.

Academic Challenge

ActiveLearning

Student Staff

Interactions


SupportiveLearning

Environment


0

10

20

30

40

50

60

70

80

Male

Female

01020304050607080

46.949.6

39.4 40.7

22.5 21.726.4

29.4

53.8 54.2

40.4

45.4


Figure 4.7.1.2 Outcomes - Gender (Overall)

4.7.2 Age

Figure 4.7.2.1 presents scores for engagement indices by age for all years of study, all modes of study and all institution-types. The engagement scores for students aged 19 and under are lower than for all other ages, apart from Supportive Learning Environment and Enriching Educational Experiences. This may reflect the more outgoing and social nature of the higher education experience often experienced by those in the earlier years of study. The notably lower scores for those aged 19 and under in Academic Challenge, Active Learning, Work-Integrated Learning, and particularly in Student-Staff Interactions, suggests a level of relative disengagement amongst a group who continue to represent a majority of new entrants to higher education. These findings are worthy of further exploration at system level and of focussed attention at institution level. The scores for Enriching Educational Experiences decline according to age group from 20-24.

01020304050607080

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

Male

Female

59.762.5 60.8 62.1

39.143.8

38.941.6

66.4 67.6


Figure 4.7.2.1 Engagement - Age (Overall)

Figure 4.7.2.2 presents scores for Outcomes indices by age for all years of study, all modes of study and all institution-types. For students aged more than 30, the scores are generally lower for Outcomes indices, other than for Overall Satisfaction. The scores for Career Readiness are lowest for students aged 50 and over.

Figure 4.7.2.2 Outcomes - Age (Overall)

19 and Under

20-24

25-29

Academic Challenge

ActiveLearning

Student Staff

Interactions


SupportiveLearning

Environment


0

10

20

30

40

50

60

70

80

01020304050607080

43.3

49.7

51.3

50.0

49.6

49.2

37.1 41

.742

.239

.941

.037

.9

15.9

24.7

25.7

23.1

23.1

22.7 27

.1 31.7

29.0

24.4

23.4

22.4

56.0

53.3

53.9

53.0

52.5

54.5

31.5

44.7 49

.849

.048

.243

.2

30-39

40-49

50 and Over

19 and Under

20-24

25-29

30-39

40-49

50 and Over



General Development

Outcomes

CareerReadiness

OverallSatisfaction

0

10

20

30

40

50

60

70

80

01020304050607080

58.2 62

.7 64.6

61.1

60.5

59.0 64

.162

.961

.360

.658

.7

42.3

43.6

42.0

38.7

36.7

38.1

35.3

45.7

44.9

40.1

36.3

27.3

66.9

64.6 67

.368

.069

.074

.5

58.2


4.7.3 Domiciliary of origin

Figure 4.7.3.1 presents the scores for engagement indices by domiciliary of origin for all years of study, all modes of study and all institution-types. The chart indicates that the engagement scores for non-Irish students are generally higher than for Irish students and this is particularly apparent in relation to Enriching Educational Experiences.

Figure 4.7.3.1 Engagement - Domicile (Overall)

Figure 4.7.3.2 illustrates outcomes index scores by domiciliary of origin for all years of study, all modes of study and all institution-types. The scores for Higher Order Thinking and for General Development Outcomes are higher for non-Irish students. Notwithstanding the generally more positive sense of engagement reflected by non-Irish students, the results show that Overall Satisfaction scores are higher for Irish students.

Figure 4.7.3.2 Outcomes - Domicile (Overall)

Irish

Non-Irish

01020304050607080

0

10

20

30

40

50

60

70

80

Academic Challenge

ActiveLearning

StudentStaff

Interactions


SupportiveLearning

Environment


48.251.2

40.0 39.9

22.024.5

27.7

34.4

54.056.7

43.140.6

01020304050607080

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

Irish

Non-Irish

61.164.2 61.6 59.6

41.544.5

40.2 42.1

67.462.3


SEcTIon 5

NATIONAl PIlOT RESUlTS IN ThE INTERNATIONAl CONTEXT

5.1. IntroductionIn this Section, results from the national pilot study are presented alongside Australasia (Australia & New Zealand combined) and, subsequently, alongside the data for participating institutions in the United States. This is possible to do because the Irish Survey of Student Engagement (ISSE) is modelled closely on the Australasian Survey of Student Engagement (AUSSE) which in turn was based on the US National Survey of Student Engagement (NSSE).

When interpreting the international comparisons, there are two key points to bear in mind.

The first refers to the general trajectory of improvement over time that has been observed in both the US and Australasia. In these systems, comparable surveys have been run on a regular basis since 2000 and 2007, respectively. The survey is not a detached objective assessment of students’ experiences of higher education. Indeed, much of the value of this survey instrument lies in the design which specifically assesses the extent to which students are engaged in empirically-derived good educational practices and what they gain from their higher education experience. Increased awareness of good practices and clarity on actual performance in relation to such practice tends to lead to enhancement of practice. This contributes to the improvement, as measured over time, observed in these other jurisdictions. Comparable data available from the 2009 AUSSE survey shows that the ‘national-level’ results improved across each of the six engagement indices by an average of 2.3 points over the three-year period between 2009 and 2012.

The second important point is that direct comparisons with the US NSSE scores are not valid due to the significantly different context. The Irish results apply to the entire publicly-funded system of higher education whereas the US results are based on a self-selecting group of higher education institutions.


Note that the data for Ireland combine the responses of first year and final year undergraduate students in order to enable international comparability.


5.2 ISSE national pilot and AUSSE 2009 and 2012, UndergraduateFigure 5.2.1 presents the scores for engagement indices for combined undergraduates for Ireland in 2013 relative to the AUSSE results in 2009 and 2012. It is of note that Student Staff Interactions and Enriching Educational Experience provide the lowest results in each of the countries. The chart demonstrates that Irish engagement scores are broadly comparable to scores from Australasia. This is particularly evident when compared to the earlier AUSSE results. Ireland displays a slightly higher score than Australasia in relation to Enriching Educational Experiences. Australasia displays a higher score than Ireland for Staff Student Interactions and Work Integrated Learning.

Figure 5.2.1 Engagement - International Comparison (Undergraduate)

Figure 5.2.2 presents the scores for Outcomes indices for combined undergraduates for Ireland in 2013 relative to AUSSE scores in 2009 and 2012. Scores for Ireland are broadly in line with Australasia in three of the five outcomes indices. The Australasia scores are substantially higher than Ireland in Higher Order Thinking and General Development Outcomes.

Academic Challenge

ActiveLearning

StudentStaff

Interactions


SupportiveLearning

Environment


0

10

20

30

40

50

60

70

80

01020304050607080

46.7

Ireland Pilot

Australasia 2009

Australasia 2012

47.9 50

.1

39.4

38.6 42

.0

21.3

23.0 26

.4 28.1

25.0

26.0

54.1

54.1

54.8

40.7 45

.2 48.3


Figure 5.2.2 Outcomes – International Comparison (Undergraduate)

Overall, it is of note that scores for engagement and outcomes indices increase from undergraduate to postgraduate for Ireland, as shown in the next charts. This is in contrast to the Australasian results.

5.3 ISSE national pilot and POSSE 2012, PostgraduateThis section presents Irish postgraduate engagement and outcome scores alongside the 2012 Australasia scores.

Figure 5.3.1 illustrates the comparison between Ireland 2013 pilot and POSSE 2012 engagement index scores. Irish and Australasian scores are broadly comparable across all indices. Ireland’s scores are marginally higher for Academic Challenge and Active Learning while Australasia’s scores are higher than Ireland for Work-Integrated Learning.

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

Ireland Pilot

Australasia 2009

Australasia 2012

59.0

65.4 69

.2

61.2 63

.1

64.6

41.7 44

.6 48.8

39.7

37.6 41

.1

66.7 68

.5 70.0


Figure 5.3.1 Engagement - International Comparison (Postgraduate)

Figure 5.3.2 illustrates the scores for outcomes indices for Ireland 2013 pilot and POSSE 2012. Australasian scores for Outcomes indices are broadly comparable to Ireland, but are higher for General Development Outcomes while the Irish scores are marginally higher than Australasia for General Learning Outcomes and Overall Satisfaction. Bearing in mind the experiences of others in terms of a trajectory of improvement post-implementation of the student survey instrument, the national pilot results for Ireland reflect very well on the levels of student engagement and student satisfaction among Irish postgraduate students.

Figure 5.3.2 Outcomes - International Comparison (Postgraduate)

Academic Challenge

ActiveLearning

StudentStaff

Interactions


SupportiveLearning

Environment


0

10

20

30

40

50

60

70

80

01020304050607080

54.9

Ireland

Australasia

51.6

42.638.6

25.3 26.8 26.727.4

53.6 52.9 51.7

57.2

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

69.671.9

62.760.8

40.645.2

42.6 42.2

68.2 66.7

Ireland

Australasia


5.4 International Comparison (Undergraduate) with the USThis section presents the scores for engagement indices for undergraduates with the Australasian (AUSSE) and the United States (NSSE) results. Again, to enable comparability, the data for Ireland combine the responses of first year and final year undergraduate students. Figures 5.4.1 illustrates that Irish engagement scores are broadly comparable to those from Australasia while scores from the US are higher. Ireland displays a slightly higher score than Australasia in relation to Enriching Educational Experiences while Australasia displays a higher score than Ireland for Student-Staff Interactions and Work Integrated Learning. Once again, it is important to note that direct comparisons with the US are not legitimate because the US scores refer to a self-selected set of participating institutions whereas the Irish and Australasian scores are reflective of the entire higher education systems in their respective jurisdictions.

Figure 5.4.1 Engagement - International Comparison (Undergraduate)

5.5 ConclusionIn general, the comparison of Irish scores with equivalent international results reflects relatively positively on the levels of student engagement and satisfaction in Irish higher education. Furthermore, results from the (“year zero”) Irish national pilot reflect strong potential for improvement. The feedback from postgraduate students was particularly strong and provides a basis to further improve the quality and international reputation of postgraduate study in Ireland in the years ahead. As noted earlier, the survey instrument focuses on, and contributes to raising awareness of, good educational practices. Although the calculated scores for Engagement and Outcomes indices can be difficult to interpret at an aggregate level, the availability of institution-level results for all participating institutions and the availability of detailed responses to individual questions provide an excellent basis for focussed enhancement of key aspects of teaching and learning. The establishment of such a high quality and comparable evidence-base to support enhancement of teaching and learning remains a key motivation for the full implementation of the ISSE.

Academic Challenge

ActiveLearning

StudentStaff

Interactions


SupportiveLearning

Environment


0

10

20

30

40

50

60

70

80

01020304050607080

46.7

Ireland

Australasia

US

50.1

56.7

39.4 42

.0

48.7

21.3

26.4

40.0

28.1

26.0

35.4

54.1

54.8

61.7

40.7

48.3


SEcTIon 6

REVIEw OF ThE 2013 PIlOT NATIONAl SURVEYA series of evaluation activities were undertaken to review experiences of the pilot project. These included discussions with institutions and with survey administrators, review workshops and the analysis of structured feedback received from institutions. A total of forty people participated in two review workshops in May 2013. The majority of institutions involved with the survey were represented at these events and some institutions were represented by multiple participants, including some student representatives. A structured review template was circulated to all institutions to gather feedback and recommendations for the future. Institutions were encouraged to complete the template after a meeting of all parties involved with local implementation of the survey in order to ensure that different institutional perspectives were reflected. In addition, the questionnaire itself included a question asking student respondents for their views on the survey.

6.1 Governance and management of the projectThe governance and management structures for the ISSE were designed to ensure wide representation of partner higher education institutions and sponsoring organisations and are outlined in section 1.3 and appendices 4 and 5. A Project Plenary Advisory Group was established with representatives from Universities, Institutes of Technology and the project co-sponsors (HEA, IOTI, IUA and USI). This Plenary Group was responsible for the overall management of the project.

6.1.1 An effective collaborative partnership

The collaborative partnership approach adopted was regarded as a vital element of delivering the pilot survey in a relatively short timescale. The collaborative model was effective and has ensured that overall decisions enabled the project to progress while taking account of the aspirations, perceptions and concerns of partner institutions and agencies. The Plenary Group met on a monthly basis from October 2012 until July 2013 and more frequently at particular stages of implementation. Members of the Survey Design and Communications and Reporting Groups also increased the frequency of meetings at appropriate times. This helped to ensure that there was a shared understanding of objectives and issues and that there was appropriate hierarchical management for the project, thus supporting delegation and escalation as required.

6.1.2 Project management and resourcing

Feedback from institutions highlighted the importance of a centralised project budget and the availability of a ‘neutral’ project management role. This ensured that support was available to address individual questions or issues and that some communication resources were provided without financial cost to individual institutions. The project manager participated in all project working groups, thus facilitating coordination of different aspects of the project. It is acknowledged that increased funding would be required to implement high quality national surveys in future years.


6.2 Feedback on the survey instrumentIn addition to the evaluation mechanisms outlined earlier, feedback from students was gathered as part of pre-testing activities with focus groups and cognitive interviews (Section 2.3). A specific question was also included in the survey itself. The main themes of feedback, received from students and from institutions, are outlined in the following sections.

6.2.1 Feedback from survey participants

A specific question was included in the 2013 pilot asking student participants to provide feedback on the survey instrument itself. A total of 1552 valid answers were received in response to this question. The main themes arising from these responses can be grouped as follows:

a) General Satisfaction

b) Length of Survey

c) Suitability for all students

d) Structure/Question Type

e) Presentation

f) Clarity

g) Completeness

a) General satisfaction

One hundred and sixty one responses addressed general satisfaction with the survey. Overall, these general comments were positive about the survey as a mechanism for students to provide feedback.

“The survey was pretty interesting and made me reflect on my own academic year and my performance during classes. Overall it’s a very good survey.”

“I think the survey is a great idea. It is very important to allow the students to voice their opinion and I would appreciate it if the survey is asked to every student before they complete their studies.”

Thirty responses to this question considered that it was important that the results of the survey be published, and that student feedback of this nature would provide information that would lead to the enhancement of the student experience.

“I think it would be important that the results would be made available not only to University staff but to students as well.”

“I’m happy with this survey so far, and I hope the information provided will actually lead to action, more so than just providing the college with information.”

b) Length of survey

There were 347 comments relating to the length of the ISSE pilot. The vast majority of these comments, 335 in total, suggested to varying degrees that the survey was very long, or that information provided prior to participation about how long the survey would take was inaccurate. Sentiment, based on those commenting on the length of the survey was broadly consistent across undergraduate first-years, undergraduate final-years and postgraduate taught students. Some responses indicated that it took 20 minutes to complete the survey. It is noteworthy that pre-testing of the instrument, in 9 institutions, indicated that most students completed the survey in 10 minutes.


c) Appropriateness of questions to different student groups

There were 448 comments relating to the capacity of the survey to serve the entire student body.

Almost 200 respondents suggested that questions to the survey should be tailored for a student’s course of academic study, particularly in reflecting the types of academic workload experience by students in different academic disciplines. Students studying Science, Engineering, and Medical or Health-related subjects were most likely to suggest such a filtering of questions in future ISSE questionnaires. Reflective of other suggestions, one respondent noted,

“Take into consideration the fact that different courses have different approaches to teaching and different levels of engagement. Perhaps have an ‘arts’ survey, ‘health sciences’, ‘engineering’ survey etc.”

Suggestions were made by respondents regarding the relevance of some questions in the survey to students completing their studies through part-time or remote-learning study. A number of respondents considered that some of the questions regarding engagement with student life on campus where focused on the full-time student experience. One student considered,

“I am a post graduate student working full time and completing a part-time Masters programme therefore not all of the questions were particularly relevant”

Similarly, 74 responses to this question suggested that it was important to differentiate between an undergraduate experience and that of a postgraduate student. Many of these comments considered that the questionnaire was most suitable for undergraduate, full-time students. One such respondent noted,

“The survey seems to be catered mostly for undergraduates. It would be useful to have a section on postgraduate research and dissertations. It would also be useful to have a section for part time students.”

d) Question response options

There were 182 responses which provided feedback on the structure of the questionnaire. These most frequently dealt with two issues, the number of options provided to respond to multiple-choice questions, and the limited opportunities to provide open-ended responses to questions.

Respondents made a number of suggestions regarding the number and scaling of multiple-choice options within the questionnaire. Many suggested the inclusion a “not applicable” option, or “this doesn’t apply to me” option. Similarly a number of students felt that they found it difficult to assess their level of engagement on some questions using the scale that was offered. Two comments reflecting this view noted,

“Some of the options are too polarised; the difference between choosing ‘Never’ and ‘Sometimes’ is vast, particularly when answer lies in the middle ground.”

“There should be a ‘once or twice/seldom’ option in between ‘never’ and ‘sometimes’.”


6.2.2 Feedback from institutions

The project sought feedback from institutions on a number of issues, including the proposed questions. The following are direct quotes (any reference to individual institutions has been removed) from feedback received pre- and post-fieldwork.

— This is a most comprehensive document. It clearly articulates the process by which the ISSE has been formulated. Useful to note the learning which has taken place from other such surveys and that emphasis is being placed on ensuring that the students interpretation of questions is addressed and that the survey is not overly long or complicated.

— The survey used is a high end tool with well proven psychometric qualities that will give exceptional data on the measured criteria.

— It has the capacity to accurately measure the activities that are proven to effect student learning in higher education and it allows national and international comparisons to be made.

— The credibility of using internationally validated survey instrument.

— The capacity to compare and benchmark institution results at institution-type level, nationally, and internationally.

— The fact that the instrument dealt with student engagement, rather than satisfaction.

— The breadth of the instrument should provide important insights across cohorts and be of particular relevance within institutions rather than across institutions.

— The survey tool is not expressly designed to address the experiences of a number of these groups. The part-time student does not share the same experience of education as a full time student. Similarly, the postgraduate student does not share many of the student engagement themes explored by this survey tool.

— The nature of the questions asked is geared toward undergraduate course work. There are similar concerns for Level 6 students. The survey questions are exploring activities that are usually experienced over a four year programme. While the majority of the questions are relevant, it may be worth assessing whether there is a bias against the Level 6 programme objectives.

6.3 Communication and promotionA series of communications resources were developed to (i) raise awareness and understanding of the project and (ii) support related activities at institutional level. These included: a website, www.studentsurvey.ie; periodic electronic updates, which were sent to senior institutional staff with an invitation to circulate more widely within institutions; hardcopy posters, which were distributed in three phases to correspond to the start, middle and end of fieldwork.

The website, www.studentsurvey.ie, was launched in early December 2012 and was designed as a central reference point for information about the survey. It was structured to provide general information about the national survey, in particular for students and staff. Documentation published on the website included background information on rationale and pre-testing, promotional materials for staff and students and the agreed project description. The website received over 41,000 visits between October 2012 and June 2013 with 36,754 of these coinciding with the period when the survey was open for participation.

The nature of the pilot, including management and governance by the developing collaborative partnership, and the associated short timeline presented some challenges to communication activities at national and institutional levels. This is acknowledged and informs key recommendations for future national surveys.


Key issues and decisions relating to implementation of the survey were considered by the Plenary Advisory Group as they arose. This was regarded as an important feature of the collaborative partnership approach to the project. The technical approach adopted for the pilot (Section 6.4) contributed to challenges encountered around communications. During the pilot, the discussion and consideration of project working groups tended to address issues as they were identified. This often led to information being circulated to institutions at short notice which reduced local opportunities to raise awareness and to encourage wider discussion among staff and students. It was not possible to provide a high-level timeline for the project in advance which had an adverse impact on planning.

The website, www.studentsurvey.ie, was promoted in emails and posters and it received a significant number of visits. However, the survey itself was not accessible via the website.

Hardcopy posters were circulated to institutions and students’ unions in three phases and were broadly welcomed as a support for other local promotion. The three phases of posters were intended to inform students of the survey (phase one), remind students during the period when the survey was open (phase two) and act as a final reminder when the survey was about to close (phase three). Feedback identified that the three sets of posters were too similar and that many readers would not have distinguished between them.

6.4 Technical deliveryA number of technical delivery approaches were used during the pilot. Two commercial survey administrators5 provided an external delivery mechanism. An approach using institutions’ local management systems was also piloted. Overall, ten institutions engaged with one of the commercial solutions, nine institutions engaged with the other commercial solution and seven institutions engaged with an internal learning management system solution.

An extract from institutions’ student records systems was used to provide certain limited contextual demographic data which were associated with student responses for high-level analysis. This approach meant that students were not required to input these data when participating in the survey, but that these data could enable analysis of subgroups, for example, by demographic and contextual factors such as gender, full-time, part-time, broad field of study. The following demographic and contextual data were used:

— Student ID6

— Institution code

— Course code

— Date of birth7

— Gender

— Student code

— Mode of study

— Programme type

— Level of study

5 EvaSys (http://www.evasys.co.uk/start.html) and QuestBack (http://www.questback.ie/) 6 Student ID values were replaced by proxy values prior to analysis of the dataset,

thereby protecting confidentiality7 Date of birth was replaced by age in years prior to analysis of the dataset, thereby

protecting confidentiality when results are analysed at local level


— Exam only

— Exchange

— Course title

— Field of study

— Domiciliary of origin

— County (if domiciliary is Ireland)

— Postal code

— Nationality

— Residence

While there were some minor variations in the detail of technical delivery at local level, institutions engaged with one of two main approaches: those institutions engaging with an external survey administrator and those utilising a solution based on their local learning management system.8

Target students, in institutions engaging with an external survey administrator, received an invitation to participate and, depending on whether they responded or opted out of further contact, a number of reminder emails from the external administrator. Each email included an external hyperlink to an individual survey. By following the link, students were presented with their own set of survey questions, thereby enabling their responses to be associated with their demographic and contextual data. This was the only route through which students could access the survey. This method required institutions to prepare an extract from their student record system and to securely submit this to the administrator. The survey administrator then took responsibility for remaining technical implementation, beginning with issuing invitations and concluding with submission of responses, matched to demographic data, to the HEA.

Institutions utilising their local learning management system presented the survey questions as an additional ‘course’. Students accessed the survey after logging into the learning management system. Institutions took local responsibility for promotion of participation, including distribution of email invitations and reminders. Following fieldwork, institutions prepared a file of responses and associated demographic data for submission to the HEA.

Each of the three technical delivery solutions used demonstrated relative strengths and weaknesses during the pilot. It is important to note that the pilot project did not constitute a procurement exercise for any technical solution but was used to inform the detailed specification for delivery of future surveys, incorporating the most effective elements of each solution. Each of the three solutions highlighted issues for consideration and these informed evaluation of the pilot survey project. The detail of the approach taken by the commercial survey administrators, given limited guidance, should not be misconstrued as defining the functionality of the respective solutions. For example, one commercial administrator provided individualised links to each institution whereby aggregated anonymised local responses could be reviewed. The other administrator did not do this for the pilot but could do so, if requested, in delivery of any future survey. One survey administrator presented the survey questions as a series of pages. This enabled students to gauge progress (‘page 1 of 10’ etc). Another survey administrator presented the questions as a single page which could be navigated by scrolling. There was an explicit rationale for this, i.e., to facilitate access on mobile devices. Institutions utilising local learning management systems to deliver the survey questions also identified strengths and weaknesses particular to their individual systems and a number expressed dissatisfaction with the resulting multi-step access process for students to access the questions. While all of these experiences were informative to evaluation of the pilot, a more consistent experience would be required in future.

8 Five institutions utilised Moodle, one utilised Blackboard and one utilised an alternative local survey instrument


Regardless of the technical delivery mechanism used, a number of key issues emerged during the pilot project. On too many occasions, technical information was circulated to institutions that required local action at short notice. This contributed to difficulties in scheduling the requested actions and limited local efforts to keep relevant personnel informed about the project. For example, in a number of institutions, particular technical personnel tasked with managing the demographic data were unfamiliar with the operation of the annual student data return to the HEA. This led to additional effort being required to fully appreciate detail of the actions required. Depending on their institution, students participated by logging onto the institution’s learning management system or by responding to a specific email sent by the survey administrator. Regardless of which system was used, students’ access to the survey was too cumbersome and may have dissuaded some potential respondents. The number of visits to the website (36,754) during fieldwork relative to the number of valid responses (12,762) may provide an indication of the potential impact of this issue. A key recommendation for future surveys is the use of a single online access point to facilitate promotion and participation.

6.5 Timing of fieldworkFieldwork for the 2013 pilot survey was planned from 4 March until 12 April. This period, which included public holidays for St. Patrick’s Day and for Easter, was chosen because it afforded sufficient time for new student entrants to experience higher education and avoided the extremely busy period towards the end of the academic year. However, this was a period when there was significant variation in institutions’ academic calendars. In many cases, the target cohorts of students were off-campus for two or more weeks during this time due to study periods, teaching practice, Easter holidays or other scheduled activities. In addition, a variety of competing demands challenged the capacity of local students’ unions to actively promote participation. These included students’ union elections and referenda in addition to locally scheduled activities. Based on feedback from institutions, it is proposed that a similar time period is appropriate to provide a balance between the length of the first-year experience of higher education and the competing demands towards the end of the academic year, but with increased flexibility to accommodate the local context. This is outlined in more detail in section 5.1

6.6 Response ratesThe national response rate was 10.9%. Response rates for individual institutions ranged from 3.8% to 21%. This report describes the experience of implementing the pilot survey and identifies a number of variables that will be removed for future iterations. These may contribute to the varied response rates. Notwithstanding this, it is evident that some institutions were more successful in promoting participation than others. In general, it appears that the creation of a local implementation team proved effective as was the active involvement of student-support services. It was also the case that, in some institutions, significant effort ‘on the ground’ had only limited positive impact on response rates.

The validity, reliability and representative nature of responses is discussed in sections 2.4 to 3.2. Data at national and institution-type levels are based on large numbers of respondents but greatest value will be derived for individual institutions when there is a sufficiently high response rate to discuss and analyse the results for particular sub-groups of the student population within the institution.


SEcTIon 7

ISSE: FROM PIlOT TO FUTURE NATIONAl SURVEYS

7.1 Discussion of the pilot studyIt is testament to the effectiveness of the collaborative partnership approach that the pilot was implemented in the planned timeframe. The project team acknowledges and appreciates the significant effort, particularly within institutions, required to reach this milestone. A series of evaluation activities were undertaken to gather feedback from institutions, external survey administrator and students. A number of key proposals were identified to facilitate implementation of a national survey in 2014. These provided a broad strategic approach to implementation of the initial post-pilot phase. The main outcomes were that:

i. The collaborative governance structure would be retained for the 2014 national survey with an increased focus on strategic issues such as how survey results could be used effectively to support enhancement

ii. Fieldwork would be conducted locally during a three-week period, chosen by individual institutions taking account of calendars and competing events, from a national survey window between early February and March 2014

iii. A single third-party survey administrator would be procured to deliver the survey to students and collate responses. This would reduce demands at institutional level, facilitating a sharper focus on promoting participation with the process for students and staff. A contracted single administrator would ensure a more consistent experience for participants completing the survey and, combined with institutions updating their student data collection notices to explicitly reference the survey, would address any remaining data protection concerns

iv. Access to the survey would be simplified for students through the use of a single web link with the minimum number of ‘clicks’ necessary.

7.2 Key recommendations for future national student surveysThe national pilot provided valuable experience and identified a number of key issues for implementation of future surveys. The main logistical issues, as identified for implementation of the survey in 2014, are outlined in section 7.1. In addition to these, a number of other matters have been considered.


Governance

The national pilot demonstrated the positive impact of a collaborative partnership approach. There is agreement from all stakeholders that this collaboration should continue to maximise added value and benefits for all. As logistical implementation becomes more routine, there is an ongoing role for an appropriate forum to act as an advisory body to guide the strategic development of the survey, to promote efficient analysis and interpretation of its results and to prompt appropriate follow up at institutional and national levels.

Communication and promotion

Terms of reference have been revised for the Communications Working Group, with a greater focus on developing a more extensive set of resources to support promotion at local level. These will be distributed to institutions in advance of local fieldwork and will be accompanied by examples of effective practice observed during the pilot, for the benefit of all. These may include process charts or checklists and encouragement to convene local implementation teams. Materials and messages intended for students will be reviewed to ensure that they are concise and to sharpen their effectiveness for that audience.

Incentives to participate

There has been much discussion about the use of incentives and there are differing views on their potential impact. The use of incentives was actively discouraged during the 2013 pilot. Nevertheless, there are examples of positive impact when Irish institutions use appropriate incentives to promote other surveys. This could also contribute to a perceived relative value for ISSE if incentives are offered for other surveys and not for ISSE. It is proposed that the national project acknowledges the benefits of institutions making decisions based on their local circumstances. The project will provide national guidelines on the nature and scale of such incentives in order to ensure a consistent position.

Feedback to students and to staff

It is critical to the success of the ISSE that significant numbers of students participate. This increases the benefits for individual institutions as they can review results at increasingly granular levels. The greatest influence on student participation is the knowledge that the student voice is being listened to.

When increasing numbers of staff appreciate the value and benefits of the data generated by the national survey, they are more likely to promote student participation. It is essential that relevant and timely feedback is provided to students and to staff. Provision of local and national results will be incorporated into any procurement of a survey contractor, thus ensuring that results are available within a short period after survey fieldwork closes nationally.


APPENdICES


APPEnDIx 1 ITEMS CONTRIBUTINg TO EACh INdEX

Academic Challenge

definition: The extent to which expectations and assessments challenge students to learn

Items

— Worked harder than you thought you could to meet a teacher’s / tutor’s standards or expectations

— Analysing the basic elements of an idea, experience or theory

— Synthesising and organising ideas, information or experiences

— Making judgements about value of information, arguments or methods

— Applying theories or concepts to practical problems or in new situations

— Number of assigned textbooks, books or book-length packs of subject readings

— Number of written assignments of fewer than 1,000 words

— Number of written assignments of between 1,000 and 5,000 words

— Number of written assignments of more than 5,000 words

— Preparing for class (e.g. studying, reading, writing, doing homework or lab work, analyzing data, rehearsing and other academic activities)

— Spending significant amounts of time on studying and on academic work

Active learning

definition: Students’ efforts to actively construct knowledge

Items

— Asked questions or contributed to discussions in class or online

— Made a class or online presentation

— Worked with other students inside class to prepare assignments

— Worked with other students outside class to prepare assignments

— Tutored or taught other university students (paid or voluntary)

— Participated in a community-based project (e.g. volunteering) as part of your study

— Discussed ideas from your readings or classes with others outside class


Student Staff Interactions

Definition: The level and nature of students’ contact and interactions with teaching staff

Items

— Discussed your grades or assignments with teaching staff

— Talked about your career plans with teaching staff or advisors

— Discussed ideas from your readings or classes with teaching staff outside class

— Received prompt written or oral feedback from teachers on academic performance

— Worked with teaching staff on activities other than coursework (e.g. committees, orientation, student organisations etc.)

— Work on a research project with a staff member outside of coursework requirements


Definition: Students’ participation in broadening educational activities

Items

— Used an online learning system to discuss or complete an assignment e.g. Moodle, Blackboard

— Had conversations with students of a different ethnicity/nationality than your own

— Had conversations with students who are very different to you in terms of their religious beliefs, political opinions or personal values

— Encouraging contact among students from different economic, social and ethnic/national backgrounds

— Community service or volunteer work

— Internship, fieldwork or clinical placement

— Participate in a study group or learning community

— Study a foreign language

— Study abroad or student exchange

— Culminating final-year experience (e.g. honours thesis, final year project, comprehensive exam, etc)

— Independent study (e.g. outside your course)

— Participating in extracurricular activities (e.g. organisations, campus publications, student associations, clubs and societies, sports, etc.)


Supportive learning Environment

definition: Students’ feelings of support within the university community

Items

— Relationships with other students

— Relationships with teaching staff

— Relationships with administrative personnel and services

— Providing support to succeed academically

— Helping cope with non-academic responsibilities

— Providing support to socialize

work Integrated learning

definition: Integration of employment-focused work experiences into study

Items

— lended academic learning with workplace experience

— Improved knowledge and skills that will contribute to employability

— Explored how to apply learning in the workforce

— Industry placement or work experience

higher Order Thinking

definition: Participation in higher order forms of thinking

Items

— Organising and Synthesising ideas, information or experiences into new, more complex interpretations and relationships

— Making judgements about the value of information, arguments or methods (e.g. examining how others gather and interpret data and assessing the soundness of their conclusions)

— Applying theories or concepts to practical problems or in new situations

— Analysing the basic elements of an idea, problem, experience or theory, such as examining a particular case or situation in depth and considering its components

general learning Outcomes

definition: development of general competencies

Items

— Acquiring job-related or work-related knowledge and skills

— Writing clearly and effectively

— Speaking clearly and effectively

— Thinking critically and analytically

— Analysing quantitative problems

— Using computing and information technology

— Working effectively with others

— Learning effectively on your own


general development Outcomes

definition: Formation of general forms of individual and social development

Items

— Voting informally in local, state or national elections

— Understanding yourself (e.g. self reflection)

— Understanding people of other racial and ethnic backgrounds

— Solving complex real-world problems

— Developing a personal code of values and ethics

— Contributing to the welfare of your community

Career Readiness

definition: Preparation for participation in the professional workforce

Items

— Spent time keeping your CV up-to-date

— Thought about how to present yourself to employers

— Explored where to look for jobs relevant to your interests

— Used networking to source information on job opportunities

— Set career development goals and plans

Overall Satisfaction

definition: Students’ overall satisfaction with their educational experience

Items

— Overall, how would you evaluate the quality of academic advice that you have received?

— Overall how would you evaluate your entire educational experience at your institution?

— If you could start all over again would you go to the same institution?


APPEnDIx 2 VAlIdITY ANd RElIABIlITY OF ThE IRISh SURVEY OF STUdENT ENgAgEMENT: RElIABIlITY TABlES

Reliability of the Engagement Indices by Sectoral Level

Table 1 Reliability Analysis of the Engagement Indices Overall and by Sectoral Level for Undergraduates

Scale name ISSE(2013)overall

ISSE(2013)

Universities

ISSE (2013)


ISSE(2013)

colleges of Education

Academic Challenge .68 .67 .70 .62

Active Learning .61 .59 .62 .60

Student Staff Interactions .66 .59 .62 .60

Enriching Educational Experiences .65 .65 .65 .53

Supportive Learning Environment .78 .77 .78 .75

Work Integrated Learning .71 .72 .68 .72

Table 2 Reliability Analysis of Engagement Indices Overall and by Sectoral Level for Postgraduates


ISSE(2013)

Universities

ISSE (2013)


ISSE(2013)


Academic Challenge .67 .67 .71 .55

Active Learning .61 .60 .63 .51

Student Staff Interactions .66 .67 .66 .54

Enriching Educational Experiences .68 .68 .70 .44

Supportive Learning Environment .77 .77 .76 .76

Work Integrated Learning .70 .71 .62 .46


Reliability of the Outcomes Indices by Sectoral Level

Table 3 Reliability Analysis of the Outcome Indices Overall and by Sectoral Level for Undergraduates


ISSE(2013)

Universities

ISSE (2013)


ISSE(2013)


Higher Order Thinking .76 .74 .79 .76

General Learning Outcomes .85 .85 .87 .83

General Development Outcomes .84 .84 .85 .80

Career Readiness .86 .86 .85 .82

Overall Satisfaction .79 .78 .81 .76

Table 4 Reliability Analysis of the Outcome Indices Overall and by Sectoral Level for Postgraduates


ISSE(2013)

Universities

ISSE (2013)


ISSE(2013)


Higher Order Thinking .80 .79 .83 .72

General Learning Outcomes .86 .86 .87 .80

General Development Outcomes .84 .84 .84 .83

Career Readiness .88 .88 .86 .91

Overall Satisfaction .83 .82 .83 .83


Inter-Item Correlations and Corrected Item Total Correlation – Engagement Indices (Undergraduates Only)

Table 5 Inter-item Correlations and Corrected Item Total Correlation – Academic Challenge

1 2 3 4 5 6 7 8 9 10 11 Corrected Item-Total Correlation

1. Worked harder than you thought you could to meet a teacher’s standards or expectations

1 .37

2. Analysing the basic elements of an idea, experience or theory

.20 1 .46

3. Organising ideas, information or experiences

.23 .51 1 .49

4. Making judgements about value of information, arguments or methods

.20 .45 .51 1 .47

5. Applying theories or concepts to practical problems or in new situations

.19 .42 .41 .39 1 .36

6. Number of assigned textbooks, books or book-length packs of subject readings

.23 .19 .21 .27 .10 1 .38

7. Number of written assignments of fewer than 1,000 words

.001 .06 .06 .03 .04 .00 1 .08

8. Number of written assignments of between 1,000 and 5,000 words

.19 .10 .14 .19 .04 .36 .06 1 .20

9. Number of written assignments of more than 5,000 words

.14 .10 .10 .08 .24 .16 .04 .30 1 .11

10. Preparing for class .28 .07 .17 .13 .07 .25 .08 .09 .16 1 .16

11. Spending significant amounts of time on studying and on academic work

.20 .23 .20 .19 .10 .18 .05 .10 .05 .24 1 .12

Table 6 Inter-item Correlations and Corrected Item Total Correlation – Active Learning

1 2 3 4 5 6 7 Corrected Item-Total Correlation

1. Asked questions or contributed to discussions in class or online

1 .28

2. Made a class or online presentation .24 1 .36

3. Worked with other students on projects during class .10 .24 1 .40

4. Worked with other students outside class to prepare assignments

.14 .29 .42 1 .42

5. Tutored or taught other university students (paid or voluntary)

.15 .13 .20 .18 1 .27

6. Participated in a community-based project (e.g. volunteering) as part of your study

.12 .16 .09 .12 .15 1 .20

7. Discussed ideas from your readings or classes with others outside class

.21 .13 .25 .19 .13 .10 1 .30


Table 7 Inter-item Correlations and Corrected Item Total Correlation – Student Staff Interactions

1 2 3 4 5 6 Corrected Item-Total Correlation

1. Discussed your grades or assignments with teaching staff 1 .49

2. Talked about your career plans with teaching staff or advisors .44 1 .52

3. Discussed ideas from your readings or classes with teaching staff outside class

.45 .51 1 .57

4. Received prompt written or oral feedback from teachers on academic performance

.36 .25 .32 1 .34

5. Worked with teaching staff on activities other than coursework

.25 .33 .37 .18 1 .40

6. Work on a research project with a staff member outside of coursework requirements

.08 .16 .16 .05 .17 1 .18

Table 8 Inter-item Correlations and Corrected Item Total Correlation – Enriching Educational Experiences

1 2 3 4 5 6 7 8 9 10 11 12 Corrected Item-Total Correlation

1. Used an online learning system to discuss or complete an assignment

1 .12

2. Had conversations with students of a different ethnicity/background than your own

.12 1 .36

3. Had conversations with students who are very different in terms of religious beliefs, political opinions or personal values

.10 .67 1 .35

4. Internship, fieldwork or clinical placement

.03 .12 .11 1 .31

5. Study group or learning community .07 .13 .14 .25 1 .37

6. Study a foreign language .03 .17 .13 .12 .18 1 .33

7. Study abroad or student exchange .02 .12 .09 .18 .17 .43 1 .34

8. Culminating final-year experience .04 .08 .06 .22 .19 .12 .20 1 .30

9. Independent study or self-designed major

.30 .10 .11 .15 .24 .20 .19 .30 1 .33

10. Participating in extracurricular activities

.08 .09 .11 .08 .09 .09 .09 .08 .06 1 .20

11. Encouraging contact among students from different economic, social and ethnic backgrounds

.12 .25 .23 .03 .10 .06 .06 .04 .06 .10 1 .20

12. Community service or volunteer work

.07 .09 .13 .29 .27 .16 .15 .15 .17 .14 .06 1 .33


Table 9 Inter-item Correlations and Corrected Item Total Correlation – Supportive Learning Environment

1 2 3 4 5 6 7 Corrected Item-Total Correlation

1. Relationships with other students 1 .41

2. Relationships with teaching staff .40 1 .61

3. Relationships with administrative personnel and services .32 .53 1 .60

4. Relationships with support services .29 .46 .60 1 .59

5. Providing support to succeed academically .25 .45 .34 .37 1 .53

6. Helping cope with non-academic responsibilities .17 .25 .24 .28 .41 1 .42

7. Providing support to socialise .20 .19 .18 .20 .34 .52 1 .35

Inter-Item Correlations and Corrected Item Total Correlation – Outcomes Indices (Undergraduate Only)

Table 10 Inter-item Correlations and Corrected Item Total Correlation – Higher Order Thinking

1 2 3 4 Corrected Item-Total Correlation

1. Analysing the basic elements of an idea, experience or theory 1 .58

2. Synthesising and organising ideas, information or experiences .51 1 .61

3. Making judgements about value of information, arguments or methods .44 .52 1 .57

4. Applying theories or concepts to practical problems or in new situations .42 .41 .39 1 .50

Table 11 Inter-item Correlations and Corrected Item Total Correlation – General Learning Outcomes

1 2 3 4 5 6 7 8 Corrected Item-Total Correlation

1. Acquiring job-related or work-related knowledge and skills

1 .51

2. Writing clearly and effectively .38 1 .62

3. Speaking clearly and effectively .41 .65 1 .66

4. Thinking critically and analytically .37 .54 .52 1 .69

5. Analysing quantitative problems .36 .37 .40 .63 1 .59

6. Using computing and information technology .33 .28 .31 .37 .42 1 .50

7. Working effectively with others .41 .37 .51 .44 .38 .46 1 .60

8. Learning effectively on your own .33 .44 .43 .50 .38 .33 .42 1 .57


Table 12 Inter-item Correlations and Corrected Item Total Correlation – General Development Outcomes

1 2 3 4 5 6 Corrected Item-Total Correlation

1. Voting informally in local, state or national elections 1 .47

2. Understanding yourself .32 1 .63

3. Understanding people of other racial and ethnic backgrounds .40 .56 1 .67

4. Solving complex real-world problems .30 .44 .44 1 .57

5. Developing a personal code of values and ethics .39 .60 .59 .59 1 .74

6. Contributing to the welfare of your community .44 .45 .51 .51 .62 1 .67

Table 13 Inter-item Correlations and Corrected Item Total Correlation – Career Readiness

1 2 3 4 5 Corrected Item-Total Correlation

1. Spent time keeping CV up-to-date 1 .61

2. Thought about how to present yourself to potential employers .57 1 .71

3. Explored where to look for jobs relevant to your interests .54 .66 1 .76

4. Used networking to source information on job opportunities .50 .55 .66 1 .70

5. Set career development and plans .39 .49 .54 .53 1 .59

Table 14 Inter-item Correlations and Corrected Item Total Correlation – Overall Satisfaction

1 2 3 Corrected Item-Total Correlation

1. Quality of the academic advice received 1 .61

2. Entire educational experience .64 1 .72

3. Attend same institution if starting over .45 .60 1 .58


APPEnDIx 3 PARTICIPATINg INSTITUTIONSDublin City University

National University of Ireland, Galway

National University of Ireland, Maynooth

Trinity College Dublin

University College Dublin

University of Limerick

University College Cork

Athlone Institute of Technology

Cork Institute of Technology

Dublin Institute of Technology

Dundalk Institute of Technology

Galway-Mayo Institute of Technology

Institute of Art, Design and Technology, Dun Laoghaire

Institute of Technology Blanchardstown

Institute of Technology Carlow

Institute of Technology Sligo

Institute of Technology Tallaght

Institute of Technology Tralee

Letterkenny Institute of Technology

Limerick Institute of Technology

Waterford Institute of Technology

Church of Ireland College of Education

Marino Institute of Education

Mater Dei Institute of Education

St Angela’s College, Sligo

St Patrick’s College, Drumcondra


APPEnDIx 4 TERMS OF REFERENCE FOR wORkINg gROUPSThe terms of reference for working groups, as defined for the pilot, are given below. As the project moved from pilot to full initial implementation, terms of reference were reviewed and updated to ensure that they remain appropriate to the further development stages.

A4.1 Project Plenary Advisory Group

The objectives of the Plenary Group are:

i. To guide the design and structure of a national student survey that

a. builds on best practice internationally

b. utilises appropriate research methodologies and survey technologies

ii. To establish and embed a survey framework that will facilitate institutions’ implementation of the student survey, by informing and amending local policies and quality assurance procedures where appropriate

iii. To ensure key stakeholders within higher education, including institutional leaders and senior management, are informed of progress and outputs of the project

iv. To approve an agreed communication strategy to increase awareness of the project and to encourage student participation

v. To approve the structure and layout of the final report from the survey.

A4.2 Survey Design Group

The objectives of the Survey Design Group are:

i. To conduct desk-based research on international examples of national student surveys in higher education

ii. To design the survey instrument and to test its validity and reliability

iii. To identify and utilise appropriate research methodologies and survey technologies.

A4.3 Communications and Reporting Group

The objectives of the Communications and Reporting Group are:

i. To develop a communications strategy for the project

ii. To promote the benefits of engagement and participation in the survey with students and staff in higher education institutions

iii. To develop resources to support institutions’ implementation of the survey

iv. To define the structure and content of report(s) resulting from the survey.


A4.4 Technical Group

The objectives of the Technical Group are:

i. To identify the optimal process of mapping survey data to key student demographic data (from the student information system via the student identifier)

ii. To identify optimal methods to distribute the survey to students

iii. To provide guidance to technical staff within institutions as required.

A4.5 Data Analysis Group

The objectives of the Data Analysis group are:

i. To research and determine appropriate analysis techniques, leading to results that add value at national, institution-type and institutional levels

ii. To identify appropriate analysis factors and variables, taking account of the agreed outline structure for the national report of the 2013 pilot

iii. To quantify the time and resource required to undertake such analysis of collated data for inclusion in the national report

iv. To undertake analysis of national response data to test the validity and reliability of the survey instrument and its indices

v. To undertake analysis of national response data to provide graphical and textual analysis to populate the evaluation report on the 2013 pilot, according to the agreed report structure

vi. To provide reports of progress, and any issues, to the Project Plenary Group on a regular basis.


APPEnDIx 5 MEMBERShIP OF PROjECT wORkINg gROUPS

Project Plenary Advisory Group

Walter Balfe Quality and Qualifications Ireland

William Bennett Letterkenny Institute of Technology

Una Crowley National University of Ireland Maynooth

David Denieffe Institute of Technology Carlow

Jonathan Drennan University College Dublin

Michael Hall Institute of Technology Tralee

William Kelly Dublin City University

Karena Maguire Quality and Qualifications Ireland

Des Moore Institute of Technology Blanchardstown

Sarah Moore University of Limerick

Pat Morgan National University of Ireland Galway

Jim Murray Institutes of Technology Ireland

Muiris O’Connor Higher Education Authority

Cat O’Driscoll Union of Students in Ireland

Sean O’Reilly Project Manager

David O’Sullivan University College Cork

Vivienne Patterson Higher Education Authority

Amanda Piesse Trinity College Dublin

Lewis Purser Irish Universities Association

Brendan Ruddy Dublin Institute of Technology

Joseph Ryan Athlone Institute of Technology

Communications and Reporting Group

Maria Brown Institute of Technology Blanchardstown

Stephanie Donegan Institute of Technology Sligo

Carina Ginty Galway-Mayo Dublin Institute of Technology

Aisling McKenna Dublin City University


Rachel O’Connor Dublin Institute of Technology



Amanda Piesse Trinity College Dublin


Data Analysis Group


Aoife Flanagan National University of Ireland Galway

Valerie Harvey Higher Education Authority

Maura McGinn University College Dublin

Aisling McKenna Dublin City University




Survey Design Group

Stephen Cassidy Cork Institute of Technology


Alan Collins University College Cork

Rosemary Cooper Institute of Technology Tallaght

Aoife Flanagan National University of Ireland Galway

Karena Maguire Quality and Qualifications Ireland





Neil Quinlan Waterford Institute of Technology

Mark Russell Dublin Institute of Technology

Brendan Ryder Dundalk Institute of Technology

Jim Walsh National University of Ireland Maynooth

Technical Group

Niamh Brennan Trinity College Dublin

Mark Glynn Dublin City University


John McNulty University College Cork


Andrew Quirke Institute of Technology Tralee

Paul Reardon Dublin Institute of Technology


APPEnDIx 6 FIEldS OF STUdY – ISCEd EXPlAINEdThe coding for field of study is based on the International Standard Classification of Education (ISCED). ISCED has been designed as an instrument suitable for assembling, compiling, and presenting statistics of education both within countries and internationally. The system is revised and updated periodically and international consultation is underway in 2013 on the latest revision. For the purposes of existing statistics, including the ISSE, ISCED 97 is the specific classification system used.

The ISCED system is built up by classifying each educational programme by field of education and by level. The ISSE uses this classification for field of study only. Three hierarchical levels of classification are used for ISCED fields of study i.e. broad fields (1 digit), narrow fields (2 digits) and detailed fields (3 digits). Higher education institutions regularly provide statistical returns to the HEA using ISCED classification. There are specific rules, which have been agreed nationally, regarding undenominated programmes and combined programmes which utilise detailed fields. It is important that data generated from the national student survey is structured consistently with other existing data sets.

ISCED Broad fields ISCED Narrow Fields ISCED Detailed fields

0 General Programmes

01 Basic/broad, general programmes

010 Basic/broad, general programmes

08 Literacy and numeracy 080 Literacy and numeracy

09 Personal development 090 Personal development

1 Education 14 Teacher training and education science

140 Teacher training and education science (broad programmes)

142 Education science

143 Training for pre-school teachers

144 Training for teachers at basic levels

145 Training for teachers with subject specialisation

146 Training for teachers of vocational subjects

2 Humanities and Arts

21 Arts 210 Arts (broad programmes)

211 Fine arts

212 Music and performing arts

213 Audio-visual techniques and media production

214 Design

215 Craft skills

22 Humanities 220 Humanities (broad programmes)

221 Religion

222 Foreign Languages

223 Mother Tongue

225 History and archaeology

226 Philosophy and ethics



3 Social Sciences, Business And Law

31 Social and behavioural science 310 Social and behavioural science (broad programmes)

311 Psychology

312 Sociology and cultural studies

313 Political science and civics

314 Economics

32 Journalism and information 321 Journalism and reporting

322 Library, information, archive

34 Business and administration 340 Business and administration (broad programmes)

341 Wholesale and retail sales

342 Marketing and advertising

343 Finance, banking, insurance

344 Accounting and taxation

345 Management and administration

346 Secretarial and office work

347 Working life

38 Law 380 Law

4 Science, Mathematics and Computing

42 Life science 421 Biology and biochemistry

422 Environmental science

44 Physical science 440 Physical science (broad programmes)

441 Physics

442 Chemistry

443 Earth science

46 Mathematics and statistics 461 Mathematics

462 Statistics

48 Computing 481 Computer science

482 Computer use

5 Engineering, Manufacturing And Construction

52 Engineering and engineering trades

520 Engineering and engineering trades (broad programmes)

521 Mechanics and metal work

522 Electricity and energy

523 Electronics and automation

524 Chemical and process

525 Motor vehicles, ships and aircraft

54 Manufacturing and processing 540 Manufacturing and processing (broad programmes)

541 Food processing

542 Textiles, clothes, footwear, leather

543 Materials (wood, paper, plastic, glass)

544 Mining and extraction

58 Architecture and building 581 Architecture and town planning

582 Building and civil engineering



6 Agriculture and Veterinary

62 Agriculture, forestry and fishery

620 Agriculture, forestry and fishery (broad programmes)

621 Crop and livestock production

622 Horticulture

623 Forestry

624 Fisheries

64 Veterinary 641 Veterinary

7 Health and Welfare

72 Health 720 Health (broad programmes)

721 Medicine

723 Nursing and caring

724 Dental studies

725 Medical diagnostic and treatment technology

726 Therapy and rehabilitation

727 Pharmacy

76 Social services 761 Child care and youth services

762 Social work and counselling

8 Services 81 Personal services 810 Personal services (broad programmes)

811 Hotel, restaurant and catering

812 Travel, tourism and leisure

813 Sports

814 Domestic services

815 Hair and beauty services

84 Transport services 840 Transport services

850 Environmental protection (broad programmes)

85 Environmental protection 851 Environmental protection technology

852 Natural environments and wildlife

853 Community sanitation services

86 Security services 860 Security services (broad programmes)

861 Protection of persons and property

862 Occupational health and safety

863 Military and defence

9 Fields Unknown 99 Fields unknown 999 Fields unknown

ISCED classification


APPEnDIx 7 FURThER RESUlTS FROM 2013 NATIONAl PIlOTThis appendix presents additional charts to illustrate scores for specific groups of students and should be considered in the context of the main results contained in sections 4 and 5 of the report.

7.1 Institution-typeThe following charts present scores for Engagement and Outcomes indices by year/cohort for each institution-type.


Figure 7.1.1 Engagement - Year/Cohort (Universities)

Figure 7.1.2 Outcomes - Year/Cohort (Universities)

Academic Challenge

ActiveLearning

Student Staff

Interactions


SupportiveLearning

Environment


First Year

Final Year

Postgraduate Taught

01020304050607080

0

10

20

30

40

50

60

70

80

45.1

51.8 55

.4

36.1 39

.9 42.8

16.6

25.3

25.5

27.4 32

.5

27.9

55.5

53.1

53.8

32.8

45.1

51.2

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

First Year

Final Year

Postgraduate Taught

59.1 64

.1

70.7

57.8

65.2

63.1

42.3 44.0

41.4

33.5

45.2

42.3

69.2

66.2 68

.9

01020304050607080


Figure 7.1.3. Engagement - Year/Cohort (Institutes of Technology)

Figure 7.1.4. Outcomes - Year/Cohort (Institutes of Technology)

Academic Challenge

ActiveLearning

Student Staff

Interactions


SupportiveLearning

Environment


First Year

Final Year

Postgraduate Taught

0

10

20

30

40

50

60

70

80

01020304050607080

41.5

47.8 52

.2

39.7 44

.7

42.3

20.1

25.9

24.7

24.5 27

.1

25.6

55.8

51.2

52.0

37.0

48.6 52

.4

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

First Year

Final Year

Postgraduate Taught

01020304050607080

53.6 57

.5

63.8

59.0 63

.8

60.6

39.2

40.2

35.8 38

.3

45.5

46.1

67.8

61.4

6

4.0


Figure 7.1.5. Engagement - Year/Cohort (Colleges of Education)

Figure 7.1.6. Outcomes - Year/Cohort (Colleges of Education)

Academic Challenge

ActiveLearning

Student Staff

Interactions


SupportiveLearning

Environment


First Year

Final Year

Postgraduate Taught

0

10

20

30

40

50

60

70

80

01020304050607080

45.7 49

.9 55.5

37.6 40

.6

38.4

17.5

24.4

24.8

23.7

32.0

25.2

55.5

52.5 55

.1 59.9

69.3

62.6

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

First Year

Final Year

Postgraduate Taught

58.5 60

.3

73.1

59.6 62

.2

63.1

42.3 44

.0

44.1

30.6

46.7

33.0

66.4

67.2 69

.7

01020304050607080


The next charts present scores for Engagement and Outcomes indices for first year students by institution-type.

Figure 7.1.7 Engagement - Institution Type (First Year)

Figure 7.1.8. Outcomes - Institution Type (First Year)

Universities (First Year)

Institutes of Technology (First Year)

Colleges of Education (First Year)

All Institutions (First Year)

Academic Challenge

ActiveLearning

Student Staff

Interactions


SupportiveLearning

Environment


0

10

20

30

40

50

60

70

80

01020304050607080

45.1

41.5 45

.7

43.8

36.1 39

.7

37.6

37.5

16.6 20

.1

17.5

17.9

27.4

24.5

23.7 26

.2

55.5

55.8

55.5

55.6

32.8 37

.0

35.4

59.9





0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

57.159

.1

53.6 58

.5

58.3

57.8

59.0

59.6

41.242

.3

39.2 42

.3

35.1

33.5 38

.3

30.6

68.6

69.2

67.8

66.4


The next charts provide scores for Engagement and Outcomes indices for final year students by institution-type.

Figure 7.1.9. Engagement - Institution Type (Final Year)

Figure 7.1.10. Outcomes - Institution Type (Final Year)

Academic Challenge

ActiveLearning

Student Staff

Interactions


SupportiveLearning

Environment


0

10

20

30

40

50

60

70

80

50.2

39.9 44

.740

.6

41.9

25.3

25.9

24.4 25.5

32.5

27.1 32

.0

30.4

53.1

51.2

52.5

52.3

45.1 48

.6

69.3

47.451

.8

47.8 49

.9

Universities (Final Year)

Institutes of Technology (Final Year)

Colleges of Education (Final Year)

All Institutions (Final Year)

01020304050607080





0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

61.464

.1

57.5 60

.3 64.5

65.2

63.8

62.2

42.444

.0

40.2 44

.0 45.4

45.2

45.5

46.7

64.4

66.2

61.4

67.2

01020304050607080


The next charts present scores for Engagement and Outcomes indices for postgraduate students by institution type.

Figure 7.1.11. Engagement - Institution Type (Postgraduate)

Figure 7.1.12. Outcomes - Institution Type (Postgraduate)

Academic Challenge

ActiveLearning

Student Staff

Interactions


Supportive Learning

Environment


0

10

20

30

40

50

60

70

80

Universities (Postgraduate)

Institutes of Technology (Postgraduate)

Colleges of Education (Postgraduate)

All Institutions (Postgraduate)

01020304050607080

54.9

42.8

42.3

38.4 42

.6

25.5

24.7

24.8

25.3 27

.9

25.6

25.2 27

.4

53.8

52.0 55

.1

53.6

51.2

52.4

62.6

51.755

.4

52.2 55

.5

Universities (Postgraduate)

Institutes of Technology (Postgraduate)

Colleges of Education (Postgraduate)

All Institutions (Postgraduate)

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

69.6

70.7

63.8

73.1

62.7

63.1

60.6 63

.1

40.6

41.4

35.8

44.1

42.6

42.3 46

.1

33.0

68.268.9

64.0

69.7

01020304050607080


7.2 Institution-type for NFQ Level 8 and aboveThe next charts present scores for Engagement and Outcomes indices by institution type on programmes at NFQ Level 8 and above only.

Figure 7.2.1. Engagement - Institution Type (Overall) - NFQ Level 8 and above only

Figure 7.2.2. Outcomes - Institution Type (Overall) - NFQ Level 8 and above only

Academic Challenge

ActiveLearning

Student Staff

Interactions


Supportive Learning

Environment


Universities (Overall)

Institutes of Technology (Overall)

Colleges of Education (Overall)

All Institutions (Overall)

01020304050607080

0

10

20

30

40

50

60

70

8047

.8

37.8 43

.139

.1

39.2

20.3 23

.4

20.4

21.1

30.0

26.7

27.1 29

.0

54.4

53.7

54.2

54.3

37.6

43.3

64.4

40.3

48.2

46.7

47.7

Universities (Overall)

Institutes of Technology (Overall)

Colleges of Education (Overall)

All Institutions (Overall)



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

0

10

20

30

40

50

60

70

80

60.7

61.7

58.3 59.7

61.7

61.3

62.7

61.0

42.5

43.4

40.3 43

.1

39.2

38.6 41

.2

37.5

66.7

67.6

64.1 66

.9


Institution-type for NFQ Level 8 onlyThe next charts present scores for Engagement and Outcomes indices for first year students by institution type on programmes at NFQ Level 8 only.

Figure 7.2.3. Engagement - Institution Type for NFQ Level 8 only

Figure 7.2.4. Outcomes - Institution Type (First Year) - NFQ Level 8 only

Academic Challenge

ActiveLearning

Student Staff

Interactions


Supportive Learning

Environment






01020304050607080

0

10

20

30

40

50

60

70

80

44.6

35.9 40

.437

.6

37.1

15.9 19

.717

.5

16.9

27.8

24.9

23.7 26

.9

55.6

56.6

55.5

55.8

31.3 35

.7

59.9

33.8

45.2

42.7 45

.7





0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

58.5

59.5

55.5 58.5

58.4

58.0

59.3

59.6

42.0

42.8

39.5 42

.3

33.4

32.9 35

.5

30.6

68.6

69.0

67.6

66.4


The next charts present scores for Engagement and Outcomes indices for final year students by institution type on programmes at NFQ Level 8 only.

Figure 7.2.5. Engagement - Institution Type (Final Year) - NFQ Level 8 only

Figure 7.2.6. Outcomes - Institution Type (Final Year) - NFQ Level 8 only

Academic Challenge

ActiveLearning

Student Staff

Interactions


Supportive Learning

Environment


0

10

20

30

40

50

60

70

80





01020304050607080

51.5

40.0

45.9

41.2

41.7

25.4

27.1

24.6

25.8

32.6

28.4 31

.9

31.4

53.1

51.0

52.4

52.5

45.1

50.7

70.7

47.751

.950

.7

50.5





0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

63.2

64.2

61.0

61.3 65

.2

65.2

65.4

62.9

43.1

44.1

41.0 44

.3

45.7

45.3 46.6

47.1

64.7

66.0

60.9

67.7


7.3 Mode of studyThe next charts present scores for Engagement and Outcomes indices for first year students by mode of study.

Figure 7.3.1. Engagement - Mode of Study (First Year)

Figure 7.3.2. Outcomes - Mode of Study (First Year)

Academic Challenge

ActiveLearning

Student Staff

Interactions


Supportive Learning

Environment


0

10

20

30

40

50

60

70

80

Full-time(First Year)

Part-time / Remote (First Year)

01020304050607080

43.7 44.3

37.6 36.3

17.521.3

26.722.1

55.953.4

33.6

48.8

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

Full-time (First Year)

Part-time / Remote (First Year)

57.2 55.858.7

55.2

42.0

35.4 34.738.4

68.371.1


The next charts present scores for Engagement and Outcomes indices for final year students by mode of study.

Figure 7.3.3. Engagement - Mode of Study (Final Year)

Figure 7.3.4. Outcomes - Mode of Study (Final Year)

Academic Challenge

ActiveLearning

Student Staff

Interactions


Supportive Learning

Environment


0

10

20

30

40

50

60

70

80

Full-time (Final Year)

Part-time / Remote (Final Year)

01020304050607080

50.8

44.942.9

33.0

26.3

19.4

31.5

21.0

52.649.8

47.3 47.8

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

Full-time (Final Year)

Part-time / Remote (Final Year)

62.2

54.8

65.0

60.1

43.4

34.5

46.1

39.0

64.2 65.9


The next charts present scores for Engagement and Outcomes indices for postgraduate students by mode of study.

Figure 7.3.5. Engagement - Mode of Study (Postgraduate)

Figure 7.3.6. Outcomes - Mode of Study (Postgraduate)

Academic Challenge

ActiveLearning

Student Staff

Interactions


Supportive Learning

Environment


0

10

20

30

40

50

60

70

80

Full-time (Postgraduate)

Part-time / Remote (Postgraduate)

01020304050607080

56.752.7

44.340.5

27.5

22.8

30.8

23.3

54.951.9

48.7

55.3

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

Full-time (Postgraduate)

Part-time / Remote (Postgraduate)

70.4 68.764.0

61.1

43.7

36.7

46.6

37.7

67.369.2


7.4 GenderThe next charts present scores for Engagement and Outcomes indices for first year students by gender.

Figure 7.4.1. Engagement - Gender (First Year)

Figure 7.4.2. Outcomes - Gender (First Year)

Male (First Year)

Female (First Year)

Academic Challenge

ActiveLearning

Student Staff

Interactions


Supportive Learning

Environment


0

10

20

30

40

50

60

70

80

01020304050607080

42.645.0

36.9 38.1

18.6 17.3

25.1 27.3

55.1 56.1

32.937.8

Male (First Year)

Female (First Year)

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

56.3 57.8 57.9 58.7

39.043.4

33.636.7

68.4 68.8


The next charts present scores for Engagement and Outcomes indices for final year students by gender.

Figure 7.4.3. Engagement - Gender (Final Year)

Figure 7.4.4. Outcomes - Gender (Final Year)

Academic Challenge

ActiveLearning

StudentStaff

Interactions


Supportive Learning

Environment


0

10

20

30

40

50

60

70

80

Male (Final Year)

Female (Final Year)

01020304050607080

48.851.5

41.142.6

26.0 25.128.4

32.3

52.3 52.4

45.749.0

Male (Final Year)

Female (Final Year)

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

59.863.0 63.5

65.4

39.8

44.9 43.846.8

63.065.7


The next charts present scores for Engagement and Outcomes indices for postgraduate students by gender.

Figure 7.4.5. Engagement - Gender (Postgraduate)

Figure 7.4.6. Outcomes - Gender (Postgraduate)

Academic Challenge

ActiveLearning

StudentStaff

Interactions


Supportive Learning

Environment


0

10

20

30

40

50

60

70

80

Male (Postgraduate)

Female (Postgraduate)

01020304050607080

53.556.1

42.2 43.0

25.3 25.4 25.928.8

53.7 53.547.8

55.2

Male (Postgraduate)

Female (Postgraduate)

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

67.171.9

62.1 63.3

38.042.8 42.1 43.1

68.0 68.3


7.5 AgeThe next charts present scores for Engagement and Outcomes indices for first year students by age.

Figure 7.5.1. Engagement - Age (First Year)

Figure 7.5.2. Outcomes - Age (First Year)

Academic Challenge

ActiveLearning

StudentStaff

Interactions


Supportive Learning

Environment


0

10

20

30

40

50

60

70

80

01020304050607080

43.2 46

.343

.1 44.8

45.5

45.2

37.0 40

.237

.5

37.7

38.7

36.2

15.9

22.7

19.8

20.4

20.6

20.6

27.1

25.427

.9

22.8

23.3

21.7

56.0

55.7

55.0

54.7

54.7

55.9

31.4

42.3

35.5

42.3 45

.142

.5

19 and Under

20-24

25-29

30-39

40-49

50 and Over

19 and Under

20-24

25-29

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

58.2

58.3

56.0

53.7 55

.554

.1 58.2

59.5

58.4

58.8

58.9

55.0

42.3

40.042

.3

38.8

39.2

33.1 35

.2 39.5

36.5

34.9

34.3

25.4

67.0

73.1

67.0 72

.071

.7 75.0

30-39

40-49

50 and Over


7.6 Engagement by domicileThe next charts present scores for Engagement and Outcomes indices for first year students by domicile.

Figure 7.6.1. Engagement - Domicile (First Year)

Figure 7.6.2. Outcomes - Domicile (First Year)

Irish (First Year)

Non-Irish (First Year)

Academic Challenge

ActiveLearning

StudentStaff

Interactions


Supportive Learning

Environment


0

10

20

30

40

50

60

70

80

01020304050607080

43.8 44.3

37.4 37.3

17.920.2

26.030.7

55.658.2

35.632.3

Irish (First Year)

Non-Irish (First Year)

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

57.1 57.4 58.4 57.6

41.145.2

35.331.8

69.064.5


The next charts present scores for Engagement and Outcomes indices for final year students by domicile.

Figure 7.6.3. Engagement - Domicile (Final Year)

Figure 7.6.4. Outcomes - Domicile (Final Year)

Academic Challenge

ActiveLearning

StudentStaff

Interactions


Supporting Learning

Environment


0

10

20

30

40

50

60

70

80

Irish (Final Year)

Non-Irish (Final Year)

01020304050607080

50.2 50.9

42.038.7

25.6 25.630.3

37.2

52.5 53.4

47.545.5

Irish (Final Year)

Non-Irish (Final Year)

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

61.6 61.964.7

58.9

42.5 42.645.4

48.0

64.8

57.1


The next charts present scores for Engagement and Outcomes indices for postgraduate students by domicile.

Figure 7.6.5. Engagement - Domicile (Postgraduate)

Figure 7.6.6. Outcomes - Domicile (Postgraduate)

Academic Challenge

ActiveLearning

StudentStaff

Interactions


Supportive Learning

Environment


0

10

20

30

40

50

60

70

80

Irish (Postgraduate)

Non-Irish (Postgraduate)

01020304050607080

54.7 57.0

42.6 42.7

25.1 27.4 26.6

35.6

53.1

57.6

52.6

44.1

Irish (Postgraduate)

Non-Irish (Postgraduate)

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

69.6 71.0

62.8 61.7

40.1

45.242.2

46.9

68.763.8


7.7 Further international comparisonThe following charts provide more detailed international comparisons with Australasia and the United States

Readers should note carefully the commentary in section 5.1 of this report which discusses the comparability of US results with Ireland and Australasia.

Figure 7.7.1. Engagement - International Comparison (First Year)

Figure 7.7.2. Outcomes - International Comparison (First Year)

Note that Work Integrated Learning and the Outcomes indices were developed by AUSSE and do not feature in the US.

Ireland (First Year)

Australasia (First Year)

US (First Year)

Academic Challenge

ActiveLearning

StudentStaff

Interactions


Supportive Learning

Environment


01020304050607080

0

10

20

30

40

50

60

70

80

43.8 48

.8 54.5

37.5 39.5 44

.2

17.9 23

.5

35.9

26.2

23.1 28

.4

55.6

56.4

63.4

35.4

43.2

Ireland (First Year)

Australasia (First Year)

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

57.1

67.1

58.362.1

41.2

47.1

35.1 36.5

68.672.3


Figure 7.7.3. Engagement - International Comparison (Final / Later Year)

Figure 7.7.4. Outcomes - International Comparison (Final / Later Year)

Note that Work Integrated Learning and the Outcomes indices were developed by AUSSE and do not feature in the US.

Academic Challenge

ActiveLearning

StudentStaff

Interactions


Supportive Learning

Environment


0

10

20

30

40

50

60

70

80

01020304050607080

50.2

Ireland (Final Year)

Australasia (Later Year)

US (Later Year)

51.0

58.4

41.9 44

.0

52.1

25.5 28

.6

42.9

30.4

28.3

40.4

52.3

53.6

60.5

47.4 52

.3

Ireland (Final Year)

Australasia (Later Year)

0

10

20

30

40

50

60

70

80



General Development

Outcomes

CareerReadiness

OverallSatisfaction

01020304050607080

61.4

70.8

64.5 66.5

42.4

50.145.4 44.6

64.468.3


APPEnDIx 8 QUESTIONS USEd FOR PIlOT

1 In your experience at your institution during the current academic year, about how often have you done each of the following?

Never Sometimes Often Very often

Asked questions or contributed to discussions in class, tutorials, labs or online

Sought advice from academic staff

Made a class or online presentation

Worked hard to master difficult content

Prepared two or more drafts of an assignment before handing it in

Used library resources on campus or online

Worked on an assignment that required integrating ideas or information from various sources

Used student learning support services e.g. learning centre, computer centre, maths support

Blended academic learning with workplace experience

Included diverse perspectives (e.g. different races, religions, genders, political beliefs, etc.) in class discussions or written assignments

Attended class without completing the required preparatory work

Kept up to date with your studies

Worked with other students inside class to prepare assignments

Worked with other students outside class to prepare assignments

Put together ideas or concepts from different subjects / modules when completing assignments or during class discussions

Tutored or taught other college students (paid or voluntary)

Participated in a community-based project (e.g. volunteering) as part of your course



Used an online learning system to discuss or complete an assignment e.g Moodle, Blackboard

Used email or an online forum to communicate with teaching staff

Discussed your grades or assignments with teaching staff / tutors

Talked about your career plans with teaching staff or career advisors

Discussed ideas from your coursework or classes with teaching staff outside class

Received timely written or oral feedback from teachers/tutors on your academic performance

Worked harder than you thought you could to meet a teacher’s/tutor’s standards or expectations

Worked with teaching staff on activities other than coursework (e.g. committees, orientation, student organisations etc.)

Discussed ideas from your coursework with others outside class (e.g. students, family members, co-workers, etc.)

Had conversations with students of a different ethnicity/nationality than your own

Had conversations with students who are very different to you in terms of their religious beliefs, political opinions or personal values


2 During the current academic year, how much has your coursework emphasised the following intellectual activities?

Very little Sometimes Quite a bit Very much

Memorising facts, ideas or methods from your subjects and coursework so you can repeat them in pretty much the same form

Analysing the basic elements of an idea, problem, experience or theory, such as examining a particular case or situation in depth and considering its components


Making judgements about the value of information, arguments or methods, (e.g. examining how others gather and interpret data and assessing the soundness of their conclusions)


3 In a typical week, how many exercises, lab reports, problem sets and tutorial questions do you complete?

None 1-2 3-4 5-6 7+

Number of pieces of work that take one hour or less to complete

Number of pieces of work that take more than one hour to complete

4 During the current academic year approximately how many:

None 1-4 5-10 11-19 20+

Assigned textbooks, books, book-length packs or journal articles of subject readings have you read?

Books or journal articles (not assigned) for personal enjoyment or academic enrichment have you read?

Assignments of fewer than 1,000 words or equivalent have you completed?

Assignments of between 1,000 and 5,000 words or equivalent have you completed?

Assignments of more than 5,000 words or equivalent have you completed?


5 Which box best represents the extent to which your assessments during the current academic year have challenged you to do your best work?

Very Little Very Much

1 2 3 4 5 6 7

6 During the current academic year, about how often have you done each of the following?


Attended an art exhibition, play, dance, music, theatre or other performance

Exercised or participated in physical fitness activities

Examined the strengths and weaknesses of your own views on a topic or issue

Improved knowledge and skills that will contribute to your employability

Developed communication skills relevant to your discipline

Explored how to apply your learning in the workplace

Tried to better understand someone else’s views by imagining how an issue looks from their perspective

Learned something that changed the way you understand an issue or concept

Spent time keeping your CV up-to-date

Thought about how to present yourself to potential employers

Explored where to look for jobs relevant to your interests

Used networking to source information on job opportunities

Set career development goals and plans


7 Which of the following have you done or do you plan to do before you graduate from your institution?

Do not know about

Have not decided

Do not plan to do

Plan to do

Done

Industry placement or work experience

Community service or volunteer work

Internship, fieldwork or clinical placement

Participate in a study group or learning community

Work on a research project with a staff member outside of coursework requirements

Study a foreign language

Study abroad or student exchange

Culminating final-year experience (e.g. honours thesis, final year project, comprehensive exam, etc.)

Independent study e.g. outside your course

Consult a college careers service for advice

Hold a leadership position in a college group or the community


8 Which of these boxes best represent the quality of your relationships with people at your institution?

Relationships with other students

Unfriendly, unsupportive

sense of alienation

Friendly, Supportive,

sense of belonging

1 2 3 4 5 6 7

Relationships with teaching staff

Unavailable, unhelpful and unsympathetic

Available, helpful and sympathetic

1 2 3 4 5 6 7

Relationships with administrative personnel

Unavailable, inconsiderate

rigid

Available, considerate,

flexible

1 2 3 4 5 6 7

Relationships with support services

Unavailable, inconsiderate

rigid

Available, considerate,

flexible

1 2 3 4 5 6 7


9 About how many hours do you spend in a typical seven-day week doing each of the following?


none 1-5 6-10 11-15 16-20 21-25 26-30 30+

Working for pay on campus

none 1-5 6-10 11-15 16-20 21-25 26-30 30+

Working for pay off campus

none 1-5 6-10 11-15 16-20 21-25 26-30 30+

Participating in extracurricular activities (e.g. organisations, campus publications, student associations, clubs and societies, sports, etc.)

none 1-5 6-10 11-15 16-20 21-25 26-30 30+

Relaxing and socialising (e.g. watching TV, partying, etc.)

none 1-5 6-10 11-15 16-20 21-25 26-30 30+

Providing care for dependents living with you (e.g. parents, children, spouse, etc.)

none 1-5 6-10 11-15 16-20 21-25 26-30 30+

Managing personal business (e.g. housework, shopping, exercise, health needs, etc.)

none 1-5 6-10 11-15 16-20 21-25 26-30 30+

Travelling to campus (e.g. driving, walking, etc.)

none 1-5 6-10 11-15 16-20 21-25 26-30 30+

Being on campus, including time spent in class

none 1-5 6-10 11-15 16-20 21-25 26-30 30+

Being on campus, excluding time spent in class

none 1-5 6-10 11-15 16-20 21-25 26-30 30+


10 To what extent does your institution encourage each of the following?


Spending significant amounts of time studying and on academic work

Providing the support you need to help you succeed academically

Encouraging contact among students from different economic, social and ethnic/national backgrounds

Helping you cope with your non-academic responsibilities (e.g. work, family, etc.)

Providing the support you need to socialise

Attending campus events and activities (e.g. special speakers, cultural performances, sporting events, etc.)

Using computers in academic work

11 Has your experience at this institution contributed to your knowledge, skills and personal development in the following areas?


Acquiring job-related or work-related knowledge and skills

Writing clearly and effectively

Speaking clearly and effectively

Thinking critically and analytically

Analysing quantitative problems

Using computing and information technology

Working effectively with others

Voting in local, or national elections or referenda

Learning effectively on your own

Understanding yourself e.g. self reflection

Understanding people of other racial, ethnic and national backgrounds

Solving complex, real-world problems


12 In this academic year have you seriously considered leaving your current institution? Mark all that apply.

No, I have not considered a change

Yes, to improve career prospects

Yes, for convenience and practical reasons

Yes, for financial reasons or to reduce study costs

Yes, for personal and family reasons

Yes, to obtain better quality education

Yes, to change to a different course in another institution

Yes, for other reasons

13 What are your plans for next year? Mark all that apply.

Continue with current study

Change to another institution

Change to another programme

Leave college before finishing your programme

Leave college having completed your programme

Progress to a higher qualification


14 Which category best represents your average overall grade so far?

no results Less than 50 50-59 60-69 70-79 80-89 90-100

15 Overall, how would you evaluate the quality of academic advice that you have received?

Poor Fair Good Excellent

16 Overall, how would you evaluate your entire educational experience at your institution?

Poor Fair Good Excellent

17 Overall, how dissatisfied or satisfied are you with your programme of study?

Very Satisfied Dissatisfied Satisfied Very Dissatisfied

18 If you could start all over again, would you go to the same institution?

Definitely no Probably no Probably yes Definitely Yes


19 The following questions provide you with an opportunity to comment on your overall academic experience at your institution.


What could be done to IMPROVE how your institution engages students?

20 Finally, this is the first time that students have had the opportunity to complete the Irish National Student Survey. If you have any comments on how the survey could be improved or made more relevant to your academic experience, please include them here.

THANK YOU FOR YOUR TIME IN COMPLETING THIS SURVEY

Items used with permission from The College Student Report, National Survey of Student Engagement, Copyright 2001-13 The Trustees of Indiana University.


APPEnDIx 9 REFERENCESActs of the Oireachtas (1997). Universities Act, 1997. Stationery Office. Dublin

Acts of the Oireachtas (1999). The Qualifications (Education and Training) Act 1999. Stationery Office. Dublin

Astin A. (1993). What Matters in College?: Four Critical Years Revisited. Jossey-Bass, San Francisco

Bologna Declaration (1999). Joint Declaration of the European Ministers of Education. Bologna 19th June 1999

Campbell, C. M. & Cabrera, A. (2011). How sound is NSSE? Investigating the psychometric properties of NSSE at a public, research extensive institution. Review of Higher Education, 35

Chickering, A.W. and Gamson, Z.F. (1987) Seven Principles for Good Practice in Undergraduate Education. AAHE Bulletin

Coates, H. (2010), Development of the Australasian Survey of Student Engagement. Higher Education, 60

Department of Education and Skills (2011) National Strategy for Higher Education to 2030: Report of the Strategy Group. Department of Education and Skills, Dublin

Dillman, D. (2000). Mail and Internet surveys: The Tailored Design Method. Wiley, New York

Drennan J. (2003) Cognitive interviews; verbal data in the development and pre-testing of questionnaires. Journal of Advanced Nursing, 42

European Association for Quality in Higher Education (2009) Standards and Guidelines for Quality Assurance in the European Higher Education Area. ENQA, Helsinki

Indiana University Centre for Postsecondary Research. ‘National Survey of Student Engagement (2012). Promoting Student Learning and Institutional Improvement: Lessons from NSSE at 13’. Bloomington, Indiana

Irish Higher Education Quality Network (IHEQN), 2009. Common Principles for Student Involvement in Quality Assurance/Quality Enhancement. Dublin

Kuh, G. (2001.) The National Survey of Student Engagement: Conceptual Framework and Overview of Psychometric Properties. University Centre for Postsecondary Research. Bloomington, Indiana

Kuh, G.D. (2001) Assessing What Really Matters to Student Learning: Inside the National Survey of Student Engagement. Change

Kuh, G.D., Pace, C.R. and Vesper, N. (1997) The Development of Process Indicators to Estimate Student Gains Associated with Good Practice in Undergraduate Education. Research in Higher Education

Pace, C.R. (1984) Measuring the Quality of College Student Experiences. An Account of the Development and Use of the College Student Experience Questionnaire. Higher Education Research Institute, Los Angeles


Pace, C.R. (1990) The Undergraduates: A Report of their Activities and College Experiences in the 1980s. UCLA Graduate School of Education, Los Angeles

Pascarella E., Terenzini P. (2005). How College Affects Students: A Third Decade of Research. Jossey-Bass. San Francisco

Skilbeck M. (2001). The University Challenged: A Review of International Trends and Issues with Particular Reference to Ireland. The Higher Education Authority, Dublin

[email protected]

student survey · 2019-10-21 · 3.3.7 higher order thinking 37 3.3.8 general learning outcomes 39...

Documents