accelerating the strategic use of student data: a ... young presentation.pdf · business...

A/PROF ANNE YOUNG Director Strategy, Planning and Performance Vice-Chancellor’s Division 21 November 2014

Accelerating the strategic use of student data: a collaborative approach

The University of Newcastle

The University of Newcastle at a glance

www.newcastle.edu.au

3

Strategy, Planning and Performance unit


Main responsibilities Strategic and corporate

planning Institution-wide KPIs and trend

indicators Business intelligence (Data

warehouse, dashboards, reports)

Enrolment projections and management

Academic QA Surveys/focus groups Predictive analytics

The team Statisticians Quality assurance

experts Qualitative researchers Business analysts Data warehouse / ETL

staff Teaching and research

experience

4

The Importance of information


“The most meaningful way to differentiate your company from your competition, the best way to put distance between yourself and the crowd, is to do an outstanding job with information. How you gather, manage, and use information determines whether you win or lose."

5

During the application process

During the enrolment process

Program and course level information

Retention

Success (percent of load passed)

Completion

Engagement (attendance, library, LMS, PASS, workshops,..)

Survey data (internal, national, international)

Focus group data

Social media

Graduate outcomes

Types of student data?


6

Management Information System


Student records

HR

LMS

Course & Program

Surveys

Research

Finance

MIS (DW)

Data Silos Integration - Aggregation Holistic Perspective

Applications

Engagement

Outcomes

7

Common issues in large organisations


Multiple disparate data collections and systems

Inconsistent terminology and definitions

Reactive approach to data quality

Difficulties accessing data

Insufficient training and education about data

Silo mentality and turf wars

The Data Governance Institute www.datagovernance.com

http://www.datagovernance.com/

8

Link to Strategic Plan?


Strategy 1.3

Achieve student success and retention

above national and international

benchmark levels through the provision

of diagnostic information to inform

targeted action plans by Faculties and

Divisions.

How did we get to this point?

9

New approach in 2009 Focus on improving the student experience Student surveys moved from paper to online All units of study, every time offered M/C with open comments encouraged All comments read & coded (55,000 in 2009 alone)

Integrated quantitative and qualitative reports prepared for each unit of study (>1500 reports in 2009)

Summary reports for Executive Committee and each School and Faculty with comparative data

Reports on the web for staff and students

Ongoing meetings with Heads of School and academic staff – lots of conversations!


10

What did the students say about teaching?


• Students think of their teachers holistically

• Teaching quality is at least as important as subject knowledge to students

• A good teacher is usually able to overcome any perceived weaknesses in a course

• Students link their own motivation to the quality of teaching they experience

11

What did the students say about relevance?


• Learning is interesting when it connects study to personal aspirations and to the ‘real world’.

• Teaching that shows clear relevance between what is being taught and why it is being taught and how that will help with study, assessment, outcomes, career paths is highly valued.

12

What did the students say about structure?


• Learning works well when content is organised, themes are linked and all course elements are well integrated

• Lectures, tutorials, laboratories, practicum and other learning should all support each other

• Learning and teaching methods, assessment approaches and course content should be coherent

• Courses/programs should be designed to provide supported learning by building on content and assessment throughout the course/program

13

What did the students say about flexibility?


• Having the opportunity to learn in ways most suitable to one’s style, environment and pace

• Flexibility of delivery may be required to accommodate personal circumstances

• Learning happens in different ways, and at different speeds

14

What did the students say about assessment?


• There is, and should be, connection between assessment and learning

• Information about requirements of an assessment task is appreciated

• Getting feedback soon after tasks and other learning demonstrations aids learning

• Relevance of assessment to content, course or aspiration is often unclear to students

15

Common drivers behind passive response or resistance to change


Response rates too low to be valid Only whingeing students respond Data not relevant / out of date Student deficit Not enough time or resources Change not a priority

Having results that demonstrate improved outcomes is critical to making sustainable changes …

…from resistance to acceptance to enthusiasm.

16

Actions taken by ‘early adopters’ of change


Student forums, student centred approach Respond positively to student feedback Student support, social events Quality assurance mechanisms

strengthened Teaching improvement initiatives Peer review of teaching Actively promote excellence in teaching Focus on academic development Support and training for program and course

coordinators Better induction for casual staff More accountability Embedded career advice and work

integrated learning

17

Institution-wide strategies – Phase 1


Improvements to course and program management

Improvements to course and program design

Accommodating diversity of student cohorts

Provision of ‘future generation’ learning spaces

Academic development

Tailored support

Language support

Diagnostic tools developed

Phone calls to students who withdrew to better understand attrition

Longitudinal student focus groups to track progress

18



Predictive modelling to enable early contact and targeted support

Using historical data to develop predictive models of attrition, particularly attrition in the first few weeks of semester

Program/discipline specific where possible

Test and refine models ahead of launch

In parallel, prepare targeted strategies in collaboration with academic staff and student support staff based on key predictors

Run prediction models in Week 0 and implement targeted support

Continue to engage with students during term

Evaluate outcomes

19



Provide a range of analytical tools and act to improve retention and success

Expand the range of ‘data products’ available to staff

Integrate new sources of data into the Management Information System

Provide more ‘self-service’ access to Business Intelligence for staff

Launch a visual analytical platform to encourage engagement with data

Discuss appropriate provision of data at a local level and then how to best integrate systems for more strategic use

Monitor results and provide feedback to students and staff

Encourage more collaboration – academic and professional staff

Celebrate success!

20

Assessing Business Intelligence Maturity


central coordinated team

breadth of data (education, research, HR, finance)

awareness/role of business units

type of ‘products’ available (reports, dashboards, analytics)

who is using the outputs

contribution of users to development of outputs

data management & data quality

perceived business value

positioning in the institutional strategy

www.jiscinfonet.ac.uk/infokits/business-intelligence/measuring-success/ocu-maturity-model

http://www.jiscinfonet.ac.uk/infokits/business-intelligence/measuring-success/ocu-maturity-model

21 McKinsey Global Institute, May 2011

Analysing large data sets will be a key basis of competition … as long as the right policies and enablers are in place.

Big data: The next frontier for competition


Companies and policy makers must tackle significant hurdles to fully capture big data's potential … including a shortage of skilled analysts and managers.

How is your institution managing these challenges and accelerating the strategic use of student data ?

Thank you

ALASI 2014 Australian Learning Analytics Summer Institute Sydney, 20-21 November 2014

CRICOS Provider 00109J | www.newcastle.edu.au

DISCUSSION

accelerating the strategic use of student data: a ... young presentation.pdf · business...

Documents