engagement metrics - unsw innovations...engagement metrics concentrate on the university activities...

Never Stand Still NewSouth Innovations

EngagementMetrics

October 2013

1

University Engagement Metrics Report

Prepared by NewSouth Innovations Pty Limited

UNIVERSITY OF NEW SOUTH WALES

October 2013

2

TABLE OF CONTENTS

1. Executive Summary ...............................................................................................................................3

2. Context ..................................................................................................................................................9

3. Rationale for Engagement Metrics .................................................................................................... 11

4. Project Overview ................................................................................................................................ 14

5. Outline of project work performed ................................................................................................... 15

6. Initial Project Findings ........................................................................................................................ 20

7. Conclusion/Recommendation ........................................................................................................... 24

Annex

1. Engagement Metrics - Explanatory Guide

2. Engagement Metrics Spreadsheet Model Template

3. Pro forma Engagement Metrics of UNSW for 2012

4. Pro forma Engagement Metrics of MQU for 2012

5. Standard Form Contract

6. Minutes of Workshops

7. Workshop Slides – Research Impact conference Melbourne, held on 25 June 2013

Workshop Slides – UNSW Sydney, held on 22 August 2013

3

1. Executive Summary

The principal objective of the Engagement Metrics project was to create an appropriate set of

measures across the research spectrum to enable universities to better understand and

influence the level of engagement with research users, which in turn leads to impact in the

economy and society.

Underpinning the Engagement Metrics is the Knowledge Exchange Model (KEM). The model

illustrates some of the relationships that are relevant to the transfer of university research into

the economy and society, which lead to impact and outcomes.

Diagram 1: Knowledge Exchange Model

Engagement Metrics concentrate on the university activities that connect the research base

with the user base. Examples are included in the green highlighted rectangle in Diagram 1; in

particular, the approach uses the revenues received, associated with those engagement

activities as a proxy for engagement.

4

Five categories were used to stratify the nature of engagement for the purposes of the

Engagement Metrics exercise and represent the research spectrum from societal outcomes to

commercial returns as typified in Diagram 2 below.

Diagram 2: Typical shape of revenues across the activity / outcome spectrum

Workshops

An important aspect of the project was consultation with the sector through a series of

workshops. These sessions presented the Engagement Metrics methodology and discussed

practical aspects involved in the collection of associated information.

In total, six (6) sessions which had a focus on Engagement Metrics were held and attended by

14 Australian universities and 1 overseas university (from Malaysia).

Further details of these workshops, including representative slides, are contained in the annex

to this report.

$ Revenue

Public Good Research Making Money

Outreach income Contract & R&D Spin-off returns

Royalties/Licenses

5

Approach

To enable a university to collect the Engagement Metrics, an explanatory guide and a

spreadsheet based form were created.

The explanatory guide, annexed to this report, was designed to provide information to the

individuals who prepare the Engagement Metrics ‘return’ as to which revenues to include and

which to exclude and the suggested approach utilising information commonly available.

A spreadsheet was also created to capture the Engagement Metrics information from the

revenue notes detailed in annual reports and enable verification back to audited information.

Initial results

At the date of this report, two universities have completed the Engagement Metrics exercise

and three other universities have expressed their interest in completing the process.

Displayed below are the results for the universities that completed the Engagement metrics

return for the 2012 calendar year using the explanatory guide and the spreadsheet based form.

Chart 1: UNSW Engagement Metrics results for calendar year 2012

-

20

40

60

80

100

120

Outreach Technology andResearch

Development

Externallyfunded Contract

Research

Licensing CompanyFormation

University of New South Wales Engagement Metric Return 2012

Revenue

$ Millions

6

Chart 2: Macquarie University Engagement Metrics Results for Calendar year 2012

As envisaged, the shape of both universities’ revenue curves correlated to the typical shape

and was useful in showing the overall quantum as well as the proportion of engagement

activities across the spectrum.

Support from the Wider Sector

While this study was led by UNSW and Macquarie University, we had initially set out to involve

one other university and over the course of the project, a number of other universities

evidenced their support through volunteering to apply the Engagement Metrics method to

their institutions, a clear signal of the value those universities foresee from beginning to

measure, monitor and grow the levels of engagement across the spectrum.

Universities which have signalled their interest in applying the Engagement Metrics approach

include:

University of Melbourne;

University of Wollongong; and

Queensland University of Technology

-

5

10

15

20

25

30

Outreach Technology andResearch

Development

Externallyfunded Contract

Research

Licensing CompanyFormation

Macquarie University Engagement Metric Return 2012

Revenue

$ Millions

7

Workshops held over recent months resulted in the presentation of the Engagement Metrics

methodology to 14 universities. Reassuringly, there was a healthy amount of constructive

discussion and exploratory questions, in particular, about how the Impact Assessment and

Engagement Metrics could operate together. The overall observation was that there was

broad support for Engagement Metrics with a minimum of dissension.

Recommendation

It is recommended that consideration be given to extending the coverage of the Engagement

Metrics to other Australian universities. It is believed that the Engagement Metrics

methodology gives a meaningful, short term measure of the extent of engagement between

universities and the research-user community. The approach provides quantitative, robust,

auditable data which are a reasonable proxy for volume and quality of engagement. Third

party objectivity is assured since all the metrics are based upon demonstrated flow of revenue

and essentially the writing of a cheque by the research user. Measures of this kind are much

less susceptible to gaming than, for example, activity measures such as patent filings and

counting numbers of engagements.

The approach also enables a greater understanding of the focus and emphasis of

university/industry engagement, with granularity across different activity-types, from those

focussed on economic development, innovation and public good, to those focussed on

generating returns. The shape of this profile for the sector, or parts of the sector can help

inform policy and future initiatives by identifying areas of strength and market-failure. In

essence the data will enable better focus of policy towards those Knowledge Exchange

mechanisms best aligned with policy priorities.

We believe the robust, attributable, short-term nature of these metrics provide a significant

advantage over the Impact Assessment case-study approach, which has significant challenges

on all three of these qualities. We believe, however, that a small number of case studies to

exemplify the metrics and/or demonstrate impacts that do not feature any revenue flow would

provide a reasonable overview of the system.

8

For example, to balance the administrative effort with the overall aims and benefits, it would

be appropriate for the Engagement Metrics to be prepared annually by each university and

supported by 3 – 5 exemplifying impact case studies per annum.

Appreciation

Gratitude is expressed to all individuals involved in the Engagement Metrics project and to all

those who participated in the related workshops and consultation sessions. In particular, NSi

would like to thank everyone involved from Macquarie University, who contributed

significantly to the success of this Engagement Metrics project.

Dr Kevin Cullen CEO, NewSouth Innovations

Grant Cooley CFO, NewSouth Innovations

9

2. Context

In June 2013, New South Innovations was commissioned to undertake a research project to

examine and evaluate the establishment of a set of engagement metrics that would provide

useful information to the sector regarding the nature and extent of engagement between

universities and external stakeholders, in particular industry and government.

The research project was to involve two main aspects:

1. Data focused activities

Identify areas where the universities or others currently have data across the

engagement spectrum and those areas where data do not exist;

Disaggregate the data into different activities across the spectrum with a view to

creating profiles for different universities;

Create definitions of measures and metrics and test these within a range of

universities in terms of practical application and consistency of interpretation;

Make recommendations to the Department of Industry, Innovation, Climate

Change, Science, Research and Tertiary Education on which metrics would be

most useful, the definitions to be used and practical application of these metrics

across the sector.

2. Workshops to facilitate consultation on the development of the engagement metrics

and associated model.

Expected Deliverables

The Engagement Metrics project set out to:

Develop an implementation framework for Engagement Metrics for relevant

organisations.

Hold a series of consultative workshops that focus discussion on developing an

Engagement Metrics data collection procedure and process.

Arrange the preparation, by way of a pilot, of the developed Engagement Metrics

Return by a small number of universities.

Provide relevant information about the administrative aspects if the Engagement

Metrics Return was to be used more widely.

10

More specifically, the project was to report on the work undertaken and any initial findings,

and amongst other aspects, could include comments to the following questions:

Data capture and quality

What data on research engagement is currently being collected by the three

universities?

What challenges were involved in extracting this data?

What challenges might be involved in attempting to capture this data from all Australian

universities?

Is the data collected of uniform quality, or are some data items more or less reliable

than others?

What might be done to improve data quality?

If additional data was to be captured in future, what should this data be?

Which aspects of the data are already reported to the Australian Government (through

mechanisms such as the Higher Education Research Data Collection (HERDC) or the

National Survey of Research Commercialisation)?

Data analysis

What does the data reveal about the research engagement situations within the three

universities?

What does the data reveal about the control/non-Easy Access Intellectual Property

universities?

Which of the data items might be most suitable to be used as the basis for research

engagement metrics?

11

3. Rationale for Engagement Metrics

There has been considerable growth in commercialisation and technology transfer from

universities in recent years. The drive for this has arisen from a combination of factors: the

greater recognition by universities of the value of their contributions to the local and national

economy and getting ideas out into the marketplace; universities wanting to boost their

engagement profile; and the need for universities to diversify their sources of income.

Engagement with industry, Technology Transfer and commercialisation of intellectual property

are now recognised as a key element of universities’ activities. However, this is such a broad

and diverse area that it has proven difficult to define a set of metrics by which performance,

impact and socio-economic benefits can be assessed.

There is an array of metrics data currently in existence, but in disparate forms, and collected by

a number of stakeholders and external organisations. Over the last few years, the number of

commercialisation surveys that universities have been asked to complete has been increasing.

All of this has prompted universities to think more carefully about what information they

collect and how it is presented, since none of the current surveys are felt to fully reflect the

broad range of activities being undertaken and the benefits that they deliver.

The proposal starts from the premise that Technology Transfer activity can be carried out for a

number of reasons across a spectrum of objectives. Refer to Diagram 3 for flow of university

activities along the spectrum. At the extremes, the activities are connected with outreach and

outcome. The definitions for the purposes of this report are detailed below:

12

Outreach oriented – the activity is carried out fundamentally to deliver public benefits in terms of innovation, with the longer term objective of creating a more dynamic innovation ecosystem which will ultimately and indirectly benefit the universities. The activity will seldom lead to direct financial returns for the university, but will increase the capacity and capability of the overall economic system. The activity will generally cost money to undertake, with very little prospect of covering its own costs. In general terms this can be viewed as activities where the university acts like an agent of economic development. The key beneficiaries would be, for example, SMEs or student entrepreneurs, who would be assisted by universities to become more innovative and competitive, but who would not usually be able to pay for the cost of the assistance provided.

Outcome oriented – the activity is carried out with a view to making financial returns to the university as well as others. In outcome oriented activity the primary beneficiaries are expected to be investors, companies and the university. This is classic commercialisation of intellectual property through licensing or invested spin-out companies, where valuable intellectual property can be exploited for financial returns.

There are a number of university activities that can generally be placed at some point along this spectrum, reflecting the typical reasons for undertaking them:

Diagram 3: University Engagement Activities

Outreach Outcome Benefits to Technology & Externally Licensing Spin-out Society Research Funded Formation Development Contract Activity

Technology Development

and Contract Research

13

Previous work in the UK has shown that, in general, the volume of activity undertaken at the two ends of the spectrum tends to be relatively small. This is in spite of the fact that many policy initiatives tend to focus on pure Economic Development or pure commercialisation. In fact, the data suggest that much more engagement happens in the middle of the spectrum through consultancy, collaboration and training, as shown in diagram 4 below. This is important in terms of policy development and suggests that focus on the activities in the middle of the spectrum might be more productive.

Diagram 4: Correlation between University Engagements that generally Incur Revenue

14

4. Project Overview

The primary purpose of the University Engagement Metrics Return trial was to identify and

demonstrate the contribution that high quality research has made to the economic benefit of

society. The objective is to create an appropriate set of measures across the spectrum, analyse

past (baseline) data for each of the universities and monitor the effects of the

commercialisation of intellectual property, technology and knowledge transfer. Implicit in this

goal is to investigate the means by which these benefits may be recognised, represented and

assessed by universities and government for future engagements.

The information for the Engagement Metrics Return has been captured from the University’s

audited annual report of corresponding periods. The audited financial report has been

prepared on the basis of the Department of Education, Employment and Workplace Relation

(DEEWR) financial reporting framework. Details of the university’s audited income have been

extracted from the Statement of Comprehensive Income and corresponding notes disclosures.

The extracted income information has been exerted in the engagement metrics return per line

by line items.

The engagement metrics return was stratified into five identifiable categories to reflect an

appropriate set of measures across the spectrum of Outreach, Technology Development and

Research Development, Externally Funded Contract Research, Licensing, Company Formation

and one category of General Exclusion. The definitions of each category are outlined in

Annexure A: Explanatory Guide.

15

5. Outline of Project Work Performed

It was determined at an early stage of the project that in order to identify what would be

required of institutions when completing a process which reported Engagement Metrics to the

Department, was a pro forma questionnaire to mimic a real life exercise.

To this end, we drafted an illustrative guide which would serve as an explanatory document

and a companion document which would be used by a preparer of the Engagement Metrics

data gathering process. This guide sets out the purpose, rationale and context of the

Engagement Metrics together with more detailed definitions and instructions on how to

complete the Engagement Metrics Return. Refer to Annexure A for Explanatory Guide.

The Engagement Metrics Return was developed in the form of an excel spreadsheet. This was

considered to be a suitable format to enable a returning institution to establish spreadsheet

based mechanisms that provide the required information through linking to general ledger

sourced information. Refer to Annexure B for Engagement Metrics Spreadsheet model

template.

The information extracted from the University’s audited financial report provided past baseline

information for the Engagement Metrics Return. Its first level of information provides

summary of revenue earned by the university that is split into separate categories of the

spectrum and one general exclusion category. Table 1 below, depicts information of overall

revenue source earned by UNSW (Table 4 depicts MQU for same period). It is noted that in

2012 financial year UNSW’s major revenue source is earned through engagement in the

technology & research development and externally funded contract research. (This was also

noted for Macquarie University for externally funded contract research in the same year.)

16

Table 1: Extract of UNSW Engagement Metrics Return (First Level)

The second level of the detailed analysis is performed in order to capture the description and

nature of university engagement. Table 2 below depict information on the description and

nature of the UNSW engagement which is split into separate categories of the spectrum and

one general exclusion category (Table 5 depicts MQU data for same period). The data obtained

in Table 2 aggregate to the first level data set of Table 1.

Table 2: Extract of UNSW Engagement Metrics Return (Second Level)

17

A third level of analysis was also necessary to capture individual transaction details from the

description and nature obtained in the second level. This information was incorporated from

the University’s General Ledger accounting system. It is noted during the audit engagement,

that the university finance team prepares a detailed reconciliation of individual account codes

which are mapped to the audited financial reports. Table 3 below depicts an example of

UNSW’s detailed schedule of accounts (Table 6 depicts an example of MQU for same period).

Table 3: Example Supporting Schedule to UNSW 2012 Financial Report (Third Level)

18

Table 4: Extract of MQU Engagement Metrics return (First Level)

Table 5: Extract of MQU Engagement Metrics return (Second Level)

19

Table 6: Example Supporting Schedule to MQU 2012 Financial Report (Third Level)

The Engagement Metric framework compiled data from the university audited financial report.

Advantages of using these data are:

Data includes all revenue earned by the university during a financial period (not only

for which HERDC publish);

The level of supporting information is typically robust and able to be obtained by the

finance department;

Data is collated during the same financial period;

Data has been verified by third party for completeness; and

Administration is efficient if a particular revenue line meets category definition

criteria then the need for further analysis is not required.

20

6. Initial Project Findings

Comments are set out below in relation to specific questions that were envisaged to be

addressed through the project.

Question

Comments

What data on research engagement is currently being collected by the three universities?

All universities are reporting figures relevant to research engagement although in a different context and for different purposes. This Engagement Metrics process aims to bring previously disparate or obscure data together in a context which conveys improved information about university research engagement.

What challenges were involved in extracting this data?

Like any new reporting exercise, it was necessary to work through the existing data sources and separate relevant and irrelevant information. Being a new requirement, information was not captured with the expectation of requiring the information in the new form, and therefore as a retrospective exercise, there is understandably more involved in sourcing the information. That being said, during the process it was evident that information was readily available, and the time consuming process was a result of examining data in detail to determine whether to include or exclude the respective revenue/s. The nature of the exercise is one that quickly lends itself to spreadsheet automation through establishing formulae and data referencing to effortlessly capture the bulk of the required information. While there are limits to what could be automated, the exercise performed with a reasonable level of skill, is not considered onerous.

What challenges might be involved in attempting to capture this data from all Australian universities?

Based on the approach taken, to heavily utilise existing available information in the form of annual reports and HERDC, only a small incremental addition to existing data collection requirements is expected. This being the case, capturing the Engagement Metrics information from all Australian Universities is not considered to add significantly to the current administrative burden.

21

Question

Comments

Is the data collected of uniform quality, or are some data items more or less reliable than others?

It is expected that revenues reported by Australian universities are highly consistent. The ability of individual universities to gather the Engagement Metrics information is expected to be affected by the quality of the underlying supporting documentation the annual reports. How easy it is for one institution or another to detail down into project / purpose / activity level information from a GL account level will vary and be a large determinant of the amount of time the Engagement Metrics collection process would take.

What might be done to improve data quality?

Opportunities to improve data quality very much depend on the current state of the data. One observation made was that the descriptions used for project names are very important and in good cases reduced the need to go into further detail thereby saving time.

If additional data was to be captured in future, what should this data be?

Ideally, at the time of entering data into systems, a categorisation (or coding) specifically for the Engagement type though this is likely to result in a greater amount of cost that would be involved in sourcing the required information retrospectively.

Which aspects of the data are already reported to the Australian Government (through mechanisms such as the Higher Education Research Data Collection (HERDC) or the National Survey of Research Commercialisation (NSRC))?

Practically all of the data required for the Engagement Metrics process is already reported to the Australian Government though in a different presentational form and in some cases, in a more aggregated way. Major parts of the Engagement Metrics data were readily obtainable from supporting schedules to the financial statements with a small number requiring more extensive project level analysis.

22

Did discussions at the

workshops include

consideration of the broader

policy rationale behind the

collection of engagement

metrics beyond what is

included in the report, and the

suitability of the EM model for

addressing that rationale?

Workshop discussions included an overview of the

Knowledge Exchange Model and discussion of the inputs,

processes and outputs involved in creating knowledge

through research, and the mechanisms through which

that knowledge could lead to outcomes and impact for

the economy and society.

The workshops discussed a range of ways that measures

can provide useful information across the spectrum of

the Knowledge Exchange Model, although emphasis was

soon directed to focussing on the mechanisms that

Universities have for transferring research output to the

users and those entities which create impact.

While all workshop participants grasped the rationale

and methods associated with the Engagement Metrics

presented in the report, some held a view that

concentrating only on revenue underestimated the

outcomes that flow from university research, for

example, citations.

All agreed that there was no such thing as one “magic

measure” and that the use of revenues espoused by the

Engagement Metrics outlined in the report overcame

many issues with some other metrics collected and

framed existing data in a way that provided more

meaningful information.

How does the data that would

be collected through your

proposed process differ from

the data currently being

collected through the NSRC

and HERDC, and should the

department consider altering

the NSRC to collect this data in

future?

The Engagement Metrics proposed differ markedly from both HERDC and NSRC.

The focus of HERDC is purely on research revenues which

is broader in scope than that of Engagement Metrics

(EM). EM focuses on a smaller yet broader cross-section

of university revenues including revenues associated

with transferring (through engagement) university

created knowledge into industry, the economy and

society.

23

While the NSRC includes measures such as contract

research and research partnerships, it also covers patent

activities and resourcing. EM captures engagement

activities of universities that the NSRC potentially does

not, for example: ARC Linkage grants, Tech vouchers, and

donations received to drive engagement activities with

industry.

Both HERDC and NSRC provide useful sources of

information relevant for their intended purpose.

Engagement Metrics is not viewed as a replacement or

substitute for these collection processes, but rather a

complimentary tool to better understand activities

directed to transferring university research output into

the hands of those that drive impact.

24

7. Conclusion/Recommendation

Over the course of undertaking the project, it was clear that support for introducing

Engagement Metrics for the benefit of the sector and wider interest groups had grown

substantially. No better evidence of this exists than two participants of the last consultation

session held, University of Melbourne and Queensland University of Technology,

demonstrating their willingness to have further dialogue with a view to introducing the

methodology and completing the return in the weeks ahead.

While it is advocated that Engagement Metrics be introduced, it is not recommended as a

substitute or replacement of any existing data reporting and measures captured but rather a

supplementary or complimentary aspect.

Recommendation

It is recommended that consideration be given to extending the coverage of the Engagement

Metrics to other Australian universities. It is believed that the Engagement Metrics

methodology would complement the preparation of Impact Assessments. To balance the

administrative effort with the overall aim and benefits, it is suggested that the Engagement

Metrics could be prepared annually by each university, and dependent on size, would

additionally produce 3 to 5 impact case studies per annum.

25

Annex

1. Explanatory guide

2. Engagement Metrics spread sheet model template

3. Pro forma Engagement Metrics of UNSW for 2012

4. Pro forma Engagement Metrics of Macquarie University for 2012

5. Standard form contract

6. Minutes of workshops

7. Workshop Slides – Research Impact conference Melbourne, held 25 June 2013

Workshop Slides – UNSW Sydney, held 22 August 2013

ANNEX 1

Annexure A to Engagement Metrics report to DIICCSRTE

6 September 2013

The Guidance and University Engagement Metrics Return is designed to

assist universities to analyse the different mechanisms by which they

engage with the economy and society. The University would be further

able to identify appropriate financial measures and track these metrics

over time to expand university engagements in future.

EXPLANATORY GUIDE TO

ENGAGEMENT METRICS

1 NewSouth Innovations Pty Limited

1.0 Guidance

1.1 Introduction

There has been considerable growth in the commercialisation of Intellectual Property (IP)

and Technology Transfer from universities in recent years. The drive for this has arisen from

a combination of factors: increased recognition by universities of the value of their

contribution to the local and national economy by getting ideas out into the marketplace;

universities wanting to boost their engagement profile; and the need for universities to

diversify their sources of income. Engagement with industry, Technology Transfer and the

commercialisation of IP are now recognised as a key element of universities’ activities.

1.2 Purpose

The primary purpose of the University Engagement Metrics Return trial is to identify and

demonstrate the contribution that high quality research has made to the economic benefit of

society. The objective is to create an appropriate set of measures across the spectrum,

analyse past (baseline) data for each of the universities and monitor the effects of the

commercialisation of IP and the technology transfer. Implicit in this goal is the purpose to

investigate the means by which these benefits may be recognised, represented and

assessed by universities and government.

1.3 Definitions

1. Outreach – The University provides a broader spectrum of activities and services to

a variety of university participants by means of economic base. The activity is carried

out fundamentally to deliver benefits in terms of innovation, with the longer term

objective of creating a more dynamic innovation ecosystem which will ultimately and

indirectly benefit the universities. While these activities and services may not return

economic profit to the university, these services activities and services are

considered to be for the wellbeing of society. Income received from this category to

be recorded, would enhance the commercialisation of IP and Technology Transfer.

As such, donations for economic development from industry to support students’

innovations are to be included. Other inclusions and exclusions are:

Inclusions:

Student entrepreneurs with relation to university research and development,

small and medium enterprises (SME) support and policy advice, which are

carried out for the public good without expectation of profit, but are for the

enhancement of commercialisation of IP and Technology Transfer; and

Philanthropic income that has been provided for specific purpose or course to the

university, however, if a fund received has not been allocated for a specific

purpose to the university, then this would be excluded. For example, general

donations or bequest with no specific outcome or condition requirements.


Exclusions:

Refer Category 1 HERDC Specification;

Capital grants are those grants provided to a University to purchase an asset of a

durable nature, even if the asset is for the purpose of conducting research.

Capital grants include grants for the construction and/or upgrade or refurbishment

of buildings, centres or facilities, as well as purchase of properties or land;

Technology vouchers provided by the University (refer to Technology and

Research Development category); and

If an item is recorded in another category, then it should be excluded from this

category (to ensure the university does not double count in multiple categories).

2. Technology And Research Development - Income received for the purpose of

enhancement of innovation in technological and research development.

Inclusion:

Philanthropic income that is for specific project that is earmarked for technology

and research development to the University;

Technology vouchers provided by the University;

Research infrastructures are distinct from grants for capital grants. Grants for

research infrastructure are considered to include grants for specific and

specialised equipment which are used in the conduct of research; and

Australian Research Council funding (ARC).

Exclusion:

If item is recorded in another category, then it should be excluded from this


3. Externally Funded Contract Activity – Research and other activities funded by

Industry and Commerce. This is genuine economic activity focused on direct

economic impacts for the company or partner which leads to knowledge transfer and

increased capability in both the universities and companies.

Inclusion:

Consultancy fees - advice and research activity provided to third party

organisations. For example, paid services received by the University from the

third party organisation.

Contract research – for example, income earned from lab times provided to the

University by the third party organisations; and

Government grants income conditioned predominantly to focus on research work

which leads to the generation of Intellectual Property.


Exclusion:

If an item is recorded in another category, then it should be excluded from this

category (to ensure the University does not double count in multiple categories).

4. Licensing - The University is the legal owner of patents, copyrights and trademarks

of the Intellectual Property and the research developed within university’s individual

faculty. The University should record all income received in the year from existing or

new licences granted to companies. It should include only payments received direct

from the companies to the university in return for licensing. Do not include funding

from other bodies to support the linkage with the company or matching funding

received as a contribution to the company’s payments.

Inclusion:

Trademarks;

Patents and copyrights; and

Royalties from licenses.

Exclusion:


category (to ensure the University does not double count in multiple categories).

It is increasingly common for universities to take equity in payment or part-payment

for a licence. Income from sale of that equity can be considered to be another form of

licensing income. Therefore it should include any income received from such a sale

of equity should be included in the University Engagement Metrics Return.

5. Company Formation - University involvement in the equity partnership or associate

investments in companies. Partnership with the third party Company (s) or start up

company formation with student for the purpose of commercialisation of Intellectual

Property and technology transfer.

Inclusion:

Dividends (dividends that relate only to equity partnership should be included,

exclude dividends from the normal University’s investment policy of managed

funds).

Exclusion:



2.0 Resources Required

Audited Annual Report.

General Ledger for Revenue Accounts.


Higher Education Research Data Collection (HERDC) Return with reconciliation to

the financial accounts.

University Engagement Metrics Return to be completed. Refer Appendix 1 University

Engagement Metrics Return in separate attachment.

Access to university’s financial records for verification of contracts and agreements,

where necessary.

3.0 Approach

The compilation of the information for the University Engagement Metrics Return has been

captured from the University’s audited annual report of corresponding periods. The audited

financial report has been prepared on the basis of the Department of Industry, Innovation,

Climate Change, Science, Research and Tertiary Education (DIICCSRTE) financial reporting

framework. Details of the University’s audited Income have been extracted from the

Statement of Comprehensive Income and corresponding Notes disclosures. The extracted

income information has been exerted in the University Engagement Metrics Return per line

by line items.

The University Engagement Metrics Return has been divided into five categories to reflect

an appropriate set of measures across the spectrum of Outreach, Technology and Research

Development, Externally Funded Contract Research, Licensing, Company Formation and

one category of General Exclusion. The definitions of each category have been provided in

section 1.0 Guidance above and referred to in section 6.0 General Exclusion.

Within the Metrics Return, the following parameter has been used for selection into the

appropriate category. These are:

i. Included – this belongs outright to that category;

ii. Part-included – further analysis is required to appropriately distribute into the

categories. Refer section 7.0 Part Included for further information on how to perform

further analysis; and

iii. Excluded – this can be discharged outright from all section 1.0 Guidance categories.

Refer to section 6.0 General Exclusion for further explanation of each excluding items

from the measures across the spectrum of the University Engagement Metrics

Return.

3.1 Process Steps

Step 1: The University Engagement Metric Return will need to be filled-in with the audited

financial income amounts in the cells provided under the Column “Audited”.

Step 2: Analysis is required by the process owners (for suggested process owners refer to

Section 4.0 Process Owners) on individual line by line items. Refer to guidance

definitions for clarity on the individual category. Note, further individual transaction

detail analysis is required for the part-included items in the Metric Return. For

individual transaction analysis an example template has been provided in the

Metrics Return Excel Tab. Please copy into new excel sheet as required for other

analysis and reference as per Metric Return.


Once further analysis has been completed and information obtained, subsequently

fill-in probable cells highlighted by the dark grey for each of the categories it

belongs to.

Step 3: Assess individual line by line item for any general exclusion and fill-in for general

exclusion in the Metric Return.

4.0 Process Owners

The preparation of the University Engagement Metrics Return is expected to require the

involvement of the University Finance functions as well as finance personnel within the

Research Division, in particular, Technology Transfer Office (or similar). A suggested team

composition is as follows:

Financial Controller/Finance Manager – coordinate and supervise of the task.

Assistant Accountant/Finance Officer – assist in completion of the University

Engagement Metrics Return.

Personal-in-charge for HERDC return – assist finance team to gather financial

information.

University’s Technology Transfer Team – assist finance team to collect information.

5.0 Timetable

The University Engagement Metrics Return is to be completed in July/August of the calendar

year by the University, after issuance of the HERDC Return and issuance of the latest

audited financial reports for last financial year end are available. Refer to timetable below:

Date Description

1 June 20xx Access to audited financial report for the last financial year.

1 July 20xx Access to HERDIC Return submission data.

2nd Friday of July 20xx

University receives instructions for the commencement of University Engagement Metric Return.

Last working day in August 20xx

Submission of the University Engagement Metric Return to the Department of Industry, Innovation, Climate Change, Science, Research and Tertiary Education (DIICCSRTE).

2nd Friday of October 20xx

Report of the University Engagement Metric Return result by the Department to individual university participant.


6.0 General Exclusion

Exclusion Comment

HECS - HELP Student Payments

These are general funds provided to the student for university tuition fees, and therefore excluded.

Fees and Charges These are general funds provided to the student for university tuition fees, and therefore excluded.

Gains on Disposal of Assets This is a non occurrence income activity by the university and therefore excluded.

Higher Education Loan Program

These are general funds provided to the student for university tuition fees, and therefore excluded.

Scholarships These are general funds provided to the student for university tuition fees, and therefore excluded.

Teaching and Learning Capital Fund

Relates to normal course of the university operation to provide teaching and learning opportunity.

Education Investment Fund Relates to normal course of the university operation to provide teaching and learning opportunity.

Capital funding for the Institute Building

Income for capital projects such as building and land are also excluded. These are long term objectives of the University for sustainability management and do not constitute commercialisation of Intellectual Property and Knowledge Transfer.

Interest Income generated in the normal course of the University's cash management policy, and therefore excluded.

Distribution from Managed Funds

Income generated in the normal course of the University's investment policy, and therefore excluded.

Scholarships and Prizes These are general funds and awards provided to the student for tuitions fees and achievements. Does not relate to University Knowledge Transfer and therefore excluded.

Subscription Income generated on normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Sponsorship Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Miscellaneous Income Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Refund from ATO franking credits

Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Net Gain on Disposal of Property, Plant and Equipment



Exclusion Comment

Sale of Goods Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Printery Revenue Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Publication and Merchandise Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Developer Contributions Received


In Kind Contributions Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Lease Incentives Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Other Revenue Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Rental Charges Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Project Research Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Hospital Services Revenue Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Recoveries Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Management Fees Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Food Sales, Hotel & Retail Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Room, Academic Dress Hire Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Child Care Fees Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.

Sports and Recreation Income Income generated in the normal course of University's operations that is not related to commercialisation of Intellectual Property and technology transfer.


7.0 Part Included

For items noted as “Part Included” in the University Engagement Metrics Return, further

detailed analysis is required of the account balance through the verification of individual

transactions. After verification of the individual transactions sum amount is apportioned to

the appropriate categories. The account balance could be obtained from the University’s

financial system i.e. General Ledger. It should be noted that the amount recorded in one

category should not appear in another category. This will result in double counting, which is

not allowed within the metric reporting.

The table 1.1 below has been formulated in the University Engagement Metrics Return excel

work sheet tab for index reference. Please insert by copy and paste further index references

as required for other accounts from the Copy excel sheet tab of University Engagement

Metrics Return.

Table 1.1 Objective:

Determine whether individual transactions in the general ledger are to be included in the engagement metrics as per definition of the categories.

Work to be performed:

Individually assess transaction in the Account's General Ledger for applicability in the categories below from definition provided earlier:

1. Outreach 2. Technology and Research Development 3. Externally Funded Contract Activity 4. Licensing 5. Company Formation 6. Exclude

General Ledger Account:

Individual Descriptions Amount Included Yes/No Category

Only include sum amount in the form with Yes to the category, and No to exclude.

Transaction similar in nature could be grouped together. E.g. Income from The Department of Health and Services for individual students for a specific grant research could be grouped as one.

This document is not to be submitted with University Engagement Metrics Return. This is for University record only for apportion within the categories.


8.0 Contact Details

NewSouth Innovation Pty Limited

Gate 14 Barker Street

Level 2 Rupert Myers Building (South Wing)

UNSW Sydney NSW 2052

Australia

Contact Name: Grant Cooley

Position: Chief Financial Officer

Phone Number: 02 9385 6500

Email: [email protected]

mailto:[email protected]


9.0 Frequently Ask Questions (FAQ)

1. Which column is to be used from the audited financial reports?

The “Consolidated” column is to be used when inserting information in University

Engagement Metrics Return.

2. What if research and development is ongoing and no income is been recorded

in the university’s financial system for the year?

If the university and the third party organisation research and development is current

and no income has been recorded in the financial systems at the end of year. This

would not be included in the University Engagement Metrics Return. Only incomes

from the audited financial report for that financial year are to be included. The

university’s general purpose financial statements are prepared on the basis of

accrual accounting and would have incorporated income for the financial year.

3. What if income is to be recorded in multiple categories?

The University Engagement Metrics Return has been highlighted in dark grey for

potential selection of the categories. In some instances, the possibility of other

categories may be applicable. Professional judgement and caution is to be used

when allocating in different categories other than the probable field nominated with

help of the guidance definition.

4. What if the University’s Statement of Comprehensive Income extract and the

accounting policy Note disclosure extract cannot be matched as per the

University Engagement Metrics Return?

The University Engagement Metrics Return has been developed from the

Department of Industry, Innovation, Climate Change, Science, Research and Tertiary

Education (DIICCSRTE) financial reporting framework. All Australian universities

would have adapted the financial reporting framework as prescribed by the

DIICCSRTE. If conflicts arise between the two, use the latest DIICCSRTE financial

reporting framework and edit University Engagement Metrics Return as required and

make notes on changes made.

5. If majority of the individual transactions balance on the General Ledger (GL) is

below less than $50,000 threshold for exclusion, would this impact the amount

to be disclosed for that account balance?

The less than $50,000 materiality threshold for individual transactions has been set

for efficiency purposes. If the individual transaction amounts in the GL is less than

$50,000. Subsequently, these individual balances could be grouped together, only if,

transactions are similar in nature and would be treated as one transaction balance.

Consequently, if the individual transactions less than $50,000 do not fall into any

group, this should be excluded.


6. What if I need to add or delete index reference?

Index references have been prepared in sequential numbers in the University

Engagement Metrics Return. Any addition or deletion should not be deviate from

associated index reference number. If new references are made or existing

references been deleted, it should be added or deleted in that chronological order.

7. Why have extracted audited annual accounts have been designated for data

collection?

For the university to ascertain incomes from commercialisation of IP and knowledge

transfer for particular period, audited annual financial reports provides the paramount

conventional data source. The audited annual financial reports would incorporate all

sources of incomes of the University in the 31 December financial period. The

audited income sources have been recorded in the university accounting system,

tested for completeness and accuracy and verified by a third party auditor. Audited

financial reports of the university enhance and supplement account reconciliation

documentations from the engagement undertaken by the finance department for that

period.

ANNEX 2

University Name Annexure B - Engagement Metrics spreadsheet templateExtract Consolidated Statement of Comprehensive Income For the Year Ended 31 Dec 20XX :

Index

Reference Revenue from continuing operations

Part Included/

Excluded Outreach

Technology and

Research

Development

Externally

funded

Contract

Research Licensing

Company

Formation Excluded Total $'000 $'000 $'000 $'000 $'000 $'000 $'000

1 Australian Government Grants 1 - - - - - - 2 HELP Australian Government Payment 1 - - - - - - 3 State and Local Government financial assistance 2 - - - - - - 4 HECS - HELP Student Payments Excluded - 5 Fees and Charges Excluded - 6 Investment Revenue 3 - - - - - - 7 Royalties, trademarks and licences Part Included - 8 Consultancy and contracts 4 - - - - - - 9 Other revenue 5 - - - - - -

10 Gains on disposal of assets Excluded - 11 Investment accounted for using the equity method Part Included - 12 Other Income 5 - - - - - -

Total Revenue and Income from Continuing - - - - - -

Note 1Commonwealth Grants Scheme and Other Grants

1.01 Commonwealth Grants Scheme Part Included - - - - - - 1.02 Indigenous Support Program Part Included - 1.03 Partnership and Participation Program Part Included - 1.04 Disability Support Program Part Included - 1.05 Capital Development Pool Part Included - 1.06 Diversity and Structural Adjustment Fund Part Included - 1.07 Transitional Cost Program Part Included - 1.08 Promotion of Excellence in Learning and Teaching Part Included - 1.09 Reward Funding Part Included - 1.10 Workplace Reform Program Part Included - 1.11 Workplace Productivity Program Part Included -

Total - - - - - -

Higher Education Loan Program2.01 HECS - HELP Excluded - 2.02 FEE - HELP Excluded - 2.03 SA HELP Excluded -

Total - - - - - -

Scholarships1.12 Australian Postgraduate Awards Excluded - 1.13 International Postgraduate Research Scholarships Excluded - 1.14 Commonwealth Education Cost Scholarships Excluded - 1.15 Commonwealth Accommodation Scholarships Excluded - 1.16 Indigenous Access Scholarships Excluded -

Total Scholarships - - - - - -

Index


Part Included/

Excluded Outreach

Technology and

Research

Development

Externally

funded

Contract

Research Licensing

Company

Formation Excluded Total DIISR Research

1.17 Joint Research Engagement Program Part Included - 1.18 Research Training Scheme Part Included - 1.19 Research Infrastructure Block Grants Part Included - 1.20 Implementation Assistance Program Part Included - 1.21 Commercialisation Training Scheme Part Included - 1.22 Sustainable Research Excellence in Universities Part Included -

Total - - - - - -

Other Capital Funding1.23 Teaching and Learning Capital Fund Excluded - 1.24 Education Investment Fund Excluded -

Total - - - - - -

Australian Research Council (ARC)i) Discovery

1.25 Project Part Included - 1.26 Fellowships Part Included - 1.27 Indigenous Researchers Development Part Included -

Total Discovery - - - - - -

ii) Linkages1.28 Infrastructure Part Included - 1.29 International Part Included - 1.30 Projects Part Included -

Total Linkages - - - - - -

iii) Networks and Centres1.31 Research Networks Part Included - 1.32 Centres Part Included -

Total Networks and Centres - - - - - - Total Australian Research Council - - - - - -

Other Australian Government Financial Assistance1.33 Human services, health and aged care Part Included - 1.34 Employment and training Part Included - 1.35 Environment, sports and territories Part Included - 1.36 Social Security Part Included - 1.37 Communication, information technology and arts Part Included - 1.38 Other Part Included - 1.39 Department of Defence Part Included - 1.40 Australian Taxation Office Part Included - 1.41 AUSAID Part Included - 1.42 National Health & Medical Research Council (NHMRC) Part Included - 1.43 Higher Education Participation Programme Part Included - 1.44 Various Other Australian Government Part Included -

Total - - - - - -

TOTAL AUSTRALIAN GOVERNMENT FINANCIAL ASSISTANCE - - - - - -

Index


Part Included/

Excluded Outreach

Technology and

Research

Development

Externally

funded

Contract

Research Licensing

Company

Formation Excluded Total

Note 2State and Local Government

3.01 Research financial assistance Part Included - 3.02 Capital funding for the Institute Building Excluded - 3.03 Other financial assistance Part Included -

Total - - - - - -

Local Government financial assistance3.04 Research financial assistance Part Included -

Total - - - - - -

Total State and Local Govt. Financial Assistance - - - - - -

Note 3Investment Revenue

6.01 Interest Excluded - 6.02 Dividends Part Included - 6.03 Net Gains on sale of investments Part Included - 6.04 Distribution from managed funds Excluded -

Total - - - - - -

Note 4Consultancy and contracts

8.01 Consultancy Part Included - 8.02 Contract- Commonwealth organisations and Business enterprises Part Included - 8.03 Trust, Foundation and individuals Part Included - 8.04 Non-Commonwealth research financial assistance Part Included - 8.05 Australian Industry Part Included - 8.06 Overseas organisations Part Included - 8.07 Co-operative Research Centre Part Included - 8.08 Other organisations Part Included - 8.09 Contract research Part Included -

Total - - - - - -

Note 5Other Revenue

9.01 Donations and Bequest Part Included - 9.02 Scholarships and prizes Excluded - 9.03 Other external grants Part Included -

Total Other Revenue - - - - - -

Other Income12.01 Subscription Excluded - 12.02 Sponsorship Excluded - 12.03 Miscellaneous income Excluded - 12.04 Refund from ATO franking credits Excluded - 12.05 Net gain on disposal of property, plant and equipment Excluded - 12.06 Sale of goods Excluded -

Index


Part Included/

Excluded Outreach

Technology and

Research

Development

Externally

funded

Contract

Research Licensing

Company

Formation Excluded Total 12.07 Printery revenue Excluded - 12.08 Publication and merchandise Excluded - 12.09 Developer contributions received Excluded - 12.10 In Kind contributions Excluded - 12.11 Lease incentives Excluded - 12.12 Other revenue Excluded - 12.13 Rental Charges Excluded - 12.14 Project research Part Included - 12.15 Hospital services revenue Excluded - 12.16 Recoveries Excluded - 12.17 Management fees Excluded - 12.18 Food sales, hotel & retail Excluded - 12.19 Room, academic dress hire Excluded - 12.20 Child care fees Excluded - 12.21 Sports and recreation income Excluded -

Total Other Income - - - - - -

ANNEX 3

University of New South Wales Annexure C - Pro forma Engagement Metrics of UNSW for 2012

Extract Consolidated Statement of Comprehensive Income For the Year Ended 31 Dec 2012 :

Index


Part Included/

Excluded Outreach

Technology

and

Research

Development

Externally

funded

Contract

Research Licensing

Company


$'000 $'000 $'000 $'000 $'000 $'000 $'000

1 Australian Government Grants 1 - 26,414 53,066 - - 618,745 698,225

2 HELP Australian Government Payment 1 - - - - - 154,313 154,313

3 State and Local Government financial assistance 2 - 27,594 - - - 1,078 28,672

4 HECS - HELP Student Payments Excluded 32,077 32,077

5 Fees and Charges Excluded 488,442 488,442

6 Investment Revenue 3 - - - - - 39,937 39,937

7 Royalties, trademarks and licences Part Included 7,074 7,074

8 Consultancy and contracts 4 - - 51,555 - - 6,073 57,628

9 Other revenue 5 2,000 5,122 - - - 13,342 20,464

10 Gains on disposal of assets Excluded 3,256 3,256

11 Investment accounted for using the equity method Part Included 190 190

12 Other Income 5 - 3,870 - - - 13,530 17,400

Total Revenue and Income from Continuing 2,000 63,000 104,621 7,074 190 1,370,793 1,547,678

Note 1

Commonwealth Grants Scheme and Other Grants

1.01 Commonwealth Grants Scheme Part Included - - - - - 242,889 242,889

1.02 Indigenous Support Program Part Included 878 878

1.03 Partnership and Participation Program Part Included 4,498 4,498

1.04 Disability Support Program Part Included 189 189

1.05 Capital Development Pool Part Included -

1.06 Diversity and Structural Adjustment Fund Part Included -

1.07 Transitional Cost Program Part Included 252 252

1.08 Promotion of Excellence in Learning and Teaching Part Included 289 289

1.09 Reward Funding Part Included 1,200 1,200

1.10 Workplace Reform Program Part Included -

1.11 Workplace Productivity Program Part Included -

Total - - - - - 250,195 250,195

Higher Education Loan Program

2.01 HECS - HELP Excluded 119,934 119,934

2.02 FEE - HELP Excluded 32,658 32,658

2.03 SA HELP Excluded 1,721 1,721

Total - - - - - 154,313 154,313

Index


Part Included/

Excluded Outreach

Technology

and

Research

Development

Externally

funded

Contract

Research Licensing

Company


Scholarships

1.12 Australian Postgraduate Awards Excluded 20,847 20,847

1.13 International Postgraduate Research Scholarships Excluded 1,858 1,858

1.14 Commonwealth Education Cost Scholarships Excluded 3 3

1.15 Commonwealth Accommodation Scholarships Excluded 78 78

1.16 Indigenous Access Scholarships Excluded 18 18

Total Scholarships - - - - - 22,804 22,804

DIISR Research

1.17 Joint Research Engagement Program Part Included 31,481 31,481

1.18 Research Training Scheme Part Included 59,155 59,155

1.19 Research Infrastructure Block Grants Part Included 22,880 22,880

1.20 Implementation Assistance Program Part Included -

1.21 Commercialisation Training Scheme Part Included -

1.22 Sustainable Research Excellence in Universities Part Included 16,512 16,512

Total - - - - - 130,028 130,028

Other Capital Funding

1.23 Teaching and Learning Capital Fund Excluded -

1.24 Education Investment Fund Excluded 17,827 17,827

Total - - - - - 17,827 17,827

Australian Research Council (ARC)

i) Discovery

1.25 Project Part Included 33,641 33,641

1.26 Fellowships Part Included 17,805 17,805

1.27 Indigenous Researchers Development Part Included 447 447

Total Discovery - - - - - 51,893 51,893

ii) Linkages

1.28 Infrastructure Part Included 3,218 3,218

1.29 International Part Included -

1.30 Projects Part Included 14,493 14,493

Total Linkages - 17,711 - - - - 17,711

iii) Networks and Centres

1.31 Research Networks Part Included -

1.32 Centres Part Included 2,964 2,964

Total Networks and Centres - - - - - 2,964 2,964

Total Australian Research Council - 17,711 - - - 54,857 72,568

Index


Part Included/

Excluded Outreach

Technology

and

Research

Development

Externally

funded

Contract

Research Licensing

Company


Other Australian Government Financial Assistance

1.33 Human services, health and aged care Part Included 8,703 76,421 85,124

1.34 Employment and training Part Included -

1.35 Environment, sports and territories Part Included -

1.36 Social Security Part Included -

1.37 Communication, information technology and arts Part Included -

1.38 Other Part Included -

1.39 Department of Defence Part Included 55,641 55,641

1.40 Australian Taxation Office Part Included -

1.41 AUSAID Part Included 10,972 10,972

1.42 National Health & Medical Research Council (NHMRC) Part Included -

1.43 Higher Education Participation Programme Part Included -

1.44 Various Other Australian Government Part Included 53,066 53,066

Total - 8,703 53,066 - - 143,034 204,803

TOTAL AUSTRALIAN GOVERNMENT FINANCIAL ASSISTANCE - 26,414 53,066 - - 773,058 852,538

Note 2

State and Local Government

3.01 Research financial assistance Part Included 27,365 27,365

3.02 Capital funding for the Institute Building Excluded -

3.03 Other financial assistance Part Included 229 932 1,161

Total - 27,594 - - - 932 28,526

Local Government financial assistance

3.04 Research financial assistance Part Included 146 146

Total - - - - - 146 146

Total State and Local Govt. Financial Assistance - 27,594 - - - 1,078 28,672

Note 3

Investment Revenue

6.01 Interest Excluded 17,624 17,624

6.02 Dividends Part Included 22,169 22,169

6.03 Net Gains on sale of investments Part Included 144 144

6.04 Distribution from managed funds Excluded -

Total - - - - - 39,937 39,937

Note 4

Consultancy and contracts

8.01 Consultancy Part Included 4,682 6,071 10,753

8.02 Contract- Commonwealth organisations and Business enterprises Part Included -

Index


Part Included/

Excluded Outreach

Technology

and

Research

Development

Externally

funded

Contract

Research Licensing

Company


8.03 Trust, Foundation and individuals Part Included -

8.04 Non-Commonwealth research financial assistance Part Included -

8.05 Australian Industry Part Included -

8.06 Overseas organisations Part Included -

8.07 Co-operative Research Centre Part Included -

8.08 Other organisations Part Included -

8.09 Contract research Part Included 46,873 1.820 46,875

Total - - 51,555 - - 6,073 57,628

Note 5

Other Revenue

9.01 Donations and Bequest Part Included 2,000 5,122 4,514 11,636

9.02 Scholarships and prizes Excluded 6,706 6,706

9.03 Other external grants Part Included 2,122 2,122

Total Other Revenue 2,000 5,122 - - - 13,342 20,464

Other Income

12.01 Subscription Excluded 138 138

12.02 Sponsorship Excluded 3,674 2,867 6,541

12.03 Miscellaneous income Excluded 196 8,276 8,472

12.04 Refund from ATO franking credits Excluded 2,249 2,249

12.05 Net gain on disposal of property, plant and equipment Excluded -

12.06 Sale of goods Excluded -

12.07 Printery revenue Excluded -

12.08 Publication and merchandise Excluded -

12.09 Developer contributions received Excluded -

12.10 In Kind contributions Excluded -

12.11 Lease incentives Excluded -

12.12 Other revenue Excluded -

12.13 Rental Charges Excluded -

12.14 Project research Part Included -

12.15 Hospital services revenue Excluded -

12.16 Recoveries Excluded -

12.17 Management fees Excluded -

12.18 Food sales, hotel & retail Excluded -

12.19 Room, academic dress hire Excluded -

12.20 Child care fees Excluded -

12.21 Sports and recreation income Excluded -

Total Other Income - 3,870 - - - 13,530 17,400

ANNEX 4

Financial Institute Macquarie University Annexure D - Pro forma Engagement Metrics of Macquarie University for 2012Extract Consolidated Statement of Comprehensive Income For the Year Ended 31 Dec 2012 :

Index Reference Revenue from continuing operations

Part Included/Excluded Outreach

Technology and

Research Development

Externally funded Contract

Research Licensing Company Formation Excluded Total

$'000 $'000 $'000 $'000 $'000 $'000 $'0001 Australian Government Grants 1 721 - 11,270 - - 192,735 204,726 2 HELP Australian Government Payment 1 - - - - - 127,489 127,489 3 State and Local Government financial assistance 2 467 - 661 - - 90 1,218 4 HECS - HELP Student Payments Excluded 21,243 21,243 5 Fees and Charges Excluded 189 280,049 280,238 6 Investment Revenue 3 - - - - - 11,052 11,052 7 Royalties, trademarks and licences Part Included 575 575 8 Consultancy and contracts 4 1,095 - 10,356 - - 8,062 19,513 9 Other revenue 5 396 - 448 - - 897 1,741

10 Gains on disposal of assets Excluded 17,183 17,183 11 Investment accounted for using the equity method Part Included - 12 Other Income 5 - - 3,351 690 - 125,552 129,593

Total Revenue and Income from Continuing 2,679 - 26,275 1,265 - 784,352 814,571

Note 1Commonwealth Grants Scheme and Other Grants

1.01 Commonwealth Grants Scheme Excluded - - - - - 134,080 134,080 1.02 Indigenous Support Program Excluded 544 544 1.03 Partnership and Participation Program Excluded 1,519 1,519 1.04 Disability Support Program Excluded 856 856 1.05 Capital Development Pool Excluded - - 1.06 Diversity and Structural Adjustment Fund Excluded - - 1.07 Transitional Cost Program Excluded 36 36 1.08 Promotion of Excellence in Learning and Teaching Excluded 619 619 1.09 Reward Funding Excluded 584 584 1.10 Workplace Reform Program Excluded - - 1.11 Workplace Productivity Program Excluded - -

Total - - - - - 138,238 138,238

Higher Education Loan Program2.01 HECS - HELP Excluded 96,732 96,732 2.02 FEE - HELP Excluded 29,439 29,439 2.03 SA HELP Excluded 1,318 1,318

Total - - - - - 127,489 127,489

Scholarships1.12 Australian Postgraduate Awards Excluded 5,225 5,225 1.13 International Postgraduate Research Scholarships Excluded 448 448 1.14 Commonwealth Education Cost Scholarships Excluded 134 134 1.15 Commonwealth Accommodation Scholarships Excluded 93 93 1.16 Indigenous Access Scholarships Excluded 101 101

Total Scholarships - - - - - 6,001 6,001

DIISR Research1.17 Joint Research Engagement Program Excluded 5,841 5,841 1.18 Research Training Scheme Excluded 12,813 12,813 1.19 Research Infrastructure Block Grants Excluded 2,650 2,650 1.20 Implementation Assistance Program Excluded - - 1.21 Commercialisation Training Scheme Excluded 250- 250- 1.22 Sustainable Research Excellence in Universities Excluded 2,002 2,002

Total - - - - - 23,056 23,056

Other Capital Funding1.23 Teaching and Learning Capital Fund Excluded - 1.24 Education Investment Fund Excluded -

Total - - - - - - -

Australian Research Council (ARC)i) Discovery

1.25 Project Part Included - 9,331 9,331 1.26 Fellowships Part Included - 6,482 6,482 1.27 Indigenous Researchers Development Part Included - 73 73




Technology and




$'000 $'000 $'000 $'000 $'000 $'000 $'000Total Discovery - - - - - 15,886 15,886

ii) Linkages1.28 Infrastructure Part Included - 809 809 1.29 International Part Included - 7- 7- 1.30 Projects Part Included - 1,962 56 2,018

Total Linkages - - 1,962 - - 858 2,820

iii) Networks and Centres1.31 Research Networks Part Included - 1.32 Centres Part Included 5,169 5,169

Total Networks and Centres - - 5,169 - - - 5,169

Total Australian Research Council - - 7,131 - - 16,744 23,875

Other Australian Government Financial Assistance1.33 Human services, health and aged care Part Included - 1.34 Employment and training Part Included - 1.35 Environment, sports and territories Part Included - 1.36 Social Security Part Included - 1.37 Communication, information technology and arts Part Included - 1.38 Other Part Included 721 4,139 8,696 13,556 1.39 Department of Defence Part Included - 1.40 Australian Taxation Office Part Included - 1.41 AUSAID Part Included - 1.42 National Health & Medical Research Council (NHMRC) Part Included - 1.43 Higher Education Participation Programme Part Included - 1.44 Various Other Australian Government Part Included -

Total 721 - 4,139 - - 8,696 13,556

TOTAL AUSTRALIAN GOVERNMENT FINANCIAL ASSISTANCE 721 - 11,270 - - 320,224 332,215

Note 2State and Local Government

3.01 Research financial assistance Part Included - 3.02 Capital funding for the Institute Building Excluded - 3.03 Other financial assistance Part Included -

Total - - - - - - -

Local Government financial assistance3.04 Research financial assistance Part Included 467 661 90 1,218

Total 467 - 661 - - 90 1,218

Total State and Local Govt. Financial Assistance 467 - 661 - - 90 1,218

Note 3Investment Revenue

6.01 Interest Excluded 8,767 8,767 6.02 Dividends Part Included 2,285 2,285 6.03 Net Gains on sale of investments Part Included - 6.04 Distribution from managed funds Excluded -

Total - - - - - 11,052 11,052

Note 4Consultancy and contracts

8.01 Consultancy Part Included 851 1,228 6,591 8,670 8.02 Contract- Commonwealth organisations and Business enterprises Part Included - 8.03 Trust, Foundation and individuals Part Included - 8.04 Non-Commonwealth research financial assistance Part Included - 8.05 Australian Industry Part Included - 8.06 Overseas organisations Part Included - 8.07 Co-operative Research Centre Part Included - 8.08 Other organisations Part Included - 8.09 Contract research Part Included 244 9,128 1,471 10,843




Technology and




$'000 $'000 $'000 $'000 $'000 $'000 $'000Total 1,095 - 10,356 - - 8,062 19,513

Note 5Other Revenue

9.01 Donations and Bequest Part Included 396 448 472 1,316 9.02 Scholarships and prizes Excluded 425 425 9.03 Other external grants Part Included -

Total Other Revenue 396 - 448 - - 897 1,741

Other Income12.01 Subscription Excluded - 12.02 Sponsorship Excluded - 12.03 Miscellaneous income Excluded - 12.04 Refund from ATO franking credits Excluded - 12.05 Net gain on disposal of property, plant and equipment Excluded - 12.06 Sale of goods Excluded - 12.07 Printery revenue Excluded - 12.08 Publication and merchandise Excluded 784 784 12.09 Developer contributions received Excluded - - 12.10 In Kind contributions Excluded - - 12.11 Lease incentives Excluded - - 12.12 Other revenue Excluded 3,351 690 7,146 11,187 12.13 Rental Charges Excluded 22,437 22,437 12.14 Project research Part Included 988 988 12.15 Hospital services revenue Excluded 68,324 68,324 12.16 Recoveries Excluded 2,615 2,615 12.17 Management fees Excluded 7,660 7,660 12.18 Food sales, hotel & retail Excluded 9,951 9,951 12.19 Room, academic dress hire Excluded 1,332 1,332 12.20 Child care fees Excluded 3,840 3,840 12.21 Sports and recreation income Excluded 475 475

Total Other Income - - 3,351 690 - 125,552 129,593

ANNEX 5

z3305272

Sticky Note

Short form contract removed for confidentiality.

ANNEX 6

Annexure F to Engagement Metrics report (August 2013)

1

Workshop 1:

Held

Melbourne 25 June 2013 9am – 3.30pm

Attendees

Refer table below

Topics discussed

Recap on objective and overview of the EM model Category definitions – inclusions and exclusions Consideration of HERDC as source Practical arrangements (timetable)

Outcomes / actions arising

Development of category definitions Need to tabulate exclusions and inclusions Consideration of using HERDC or Annual report as source

Registered Attendees:

Position Organisation State

Research Coordinator James Cook University QLD

Director, Research Charles Sturt University NSW

Research Officer University of Melbourne VIC

Director of Research Services James Cook University QLD

Manager, Research Strategy and Special

Projects James Cook University QLD

Southern Cross University NSW

Dean, Research and Graduate Studies Curtin University WA

Director, Research Office Macquarie University NSW

Executive Officer Macquarie University NSW

Manager, Research and Innovations Lincoln University VIC

Director, Learning and Teaching Central Queensland University QLD


2

Education Research Centre

Portfolio Manager, Research and

Innovation University of South Australia SA

Senior Academic Advisor, Strategy and

Scholarship University of Sunshine Coast QLD

Principal Research Fellow, Partnership

and Engagement University of Ballarat VIC

Impact Project Officer CSIRO ACT

Executive Director Universiti Kebangsaan Malaysia,

AsiaEngage Selangor

Deputy Vice - Chancellor(Research) And

Vice President (Research) Charles Sturt University NSW


3

Workshop 2:

Held

Macquarie University 25 July 2013 10am – 12.30pm

Attendees

Dr K Cullen (UNSW) Mr G Cooley (UNSW) Ms T Hocking (MQ) Ms L Fleck (MQ) Ms S Chong (MQ)

Topics discussed

Recap on objective and overview of the EM model Category definitions – inclusions and exclusions Consideration of HERDC as source Practical arrangements (timetable)


Development of category definitions Need to tabulate exclusions and inclusions Consideration of using HERDC or Annual report as source


4

Workshop 3:

Held

Macquarie University 8 August 2013 2pm – 3.30pm

Attendees

Mr G Cooley (UNSW) Mr H Patel (UNSW) Ms S Chong (MQ) Ms M Stiemer (MQ)

Topics discussed

Presentation and discussion of the first draft of the Engagement Metric Return Model spreadsheet and the Annual report led approach More specific discussion around category definitions – inclusions and exclusions HERDC as secondary source. The revenue note from the Annual reports and associated supporting schedules would be the starting point of the EM model.


EM model spreadsheet provided to MQ for completion. MQ to revisit its annual report supporting work papers and capability to source project level detail. UNSW to further refine definitions and expand on explanatory information.


5

Workshop 4:

Held

University of New South Wales 22 August 2013 10am – 1.00pm

Attendees

Dr Kevin Cullen (NSi) Mr Grant Cooley (UNSW) Mr Graham Morton (NSi) Dr Jim Henderson (NSi) Mr Murray Green (UNSW) James Green - University of Melbourne James Yeates - CSIRO Troy Brown - CSIRO Hugh Nalder - QUT Bluebox Louise Fleck - Macquarie University Tori Hocking - Macquarie University Colin Melvin - Queensland University of Technology Les Rymer - Go8 Jacinta Cubis - University of Melbourne Alastair Hick – Monash University Ian Mcmahon - Go8

Topics discussed

Presentation and discussion of the first draft of the Engagement Model spreadsheet and the Annual report led approach More specific discussion around category definitions – inclusions and exclusions HERDC as secondary source. The revenue note from the Annual reports and associated supporting schedules would be the starting point of the EM model.


UNSW to further refine definitions and expand on explanatory information. For simplicity reduce spectrum categories from five to four (merge together Technology & Research Development and Externally Funded Contract Activity) Publication research by the University is not captured in the


6

Engagement Metric Return – cannot be measured in monetary value, hence could lead to underestimation of knowledge transfer by universities. The University of Melbourne and the Queensland University of Technology have agreed to be part of the pilot project trial. Group discussion agreed that source of university funds is good proxy to measure of the Industry and Government engagements.

ANNEX 7

1

Dr Kevin CullenCEONewSouth Innovations Pty Limited University of New South Wales

Assessing University Engagement and Impact

NewSouth Innovations

University Tech Transfer Measurement

Objective

To understand, assess and increase university engagement

with industry, the economy, society and community.

We must first understand the system that we are dealing

with….

2

NewSouth Innovations Knowledge Exchange Model

RESEARCH OUTPUTS

Publications

Processes

Materials

Technology

Know-how

Innovation

Skills

New

Knowledge

Researchers

Teaching

Networking/

Events

Consultancy

Professional

Development

Collaborative

Research

Contract

Research

Licensing

Company

Creation

KE CHANNELS

Knowledge Flow

Jobs

New Products

New services

Turnover

Profit

R&D expenditure

% turnover from

new products/

services

IMPACT

Start-up

Spin-out

Society

Government

Policy-makers

Entrepreneurs

Small Co’s

Big Co’s

USERS

Research

OTHER

FACTORS

OTHER

FACTORS


Key Points

The ecosystem diagram shows the inter-relationship between research and

impact. In particular, it shows the channels through which knowledge flows

happen and shows the critical importance of research-users in creating

impacts.

• The ERA has focussed on the left-hand side of the system, looking at

research outputs – created by researchers with no need to involve

research-users.

• The Impact Pilot is focussed on the right-hand end of the system, looking at

the impacts – created by research-users with no need to involve

researchers.

….neither of these really address the area where university tech transfer

happens….so let’s look at tech transfer activities.

3


There are a range of university/industry

engagement activities.....

- Are they all tech transfer/commercialisation?

- Which are most important to the University?

- Which are most important for Australia?

- Do they all make money?

....looking at the objectives for certain activities:


Studententerprise

SME networks Consultancy

Collaborative Research

Licences Spin-outs

Public

Good

Yes Yes ? No No No

Academic

Reasons

? ? Yes Yes No No

Profit NO NO ? No Yes Yes

The range of objectives

4


From the University Perspective

- At the left we are agents of economic development, seeking to

maximise public good – society and economy are the beneficiaries

- At the right we are venturers, seeking to maximise financial returns

to the University – University is the beneficiary

- In the middle, we are co-creators of knowledge – company and

university both benefit.

- These are really very different, therefore metrics are a challenge


So metrics are a challenge…

At the left hand end, success results in losing money, whereas at the right

end success results in making money. In the middle breakeven is success.

THEREFORE: Profit isn’t a good measure

At the right end success usually looks like a very large number of small

engagements, at the right hand end, success is usually from a very small

number of big deals.

THEREFORE: numbers of engagements isn’t a good measure

In the middle?

5


Issues to Discuss

- What does success look like?

If we don’t know what success looks like, how can we assess

performance?

- At the policy level, what particular types of engagement are preferred, if

any?

- What should we be measuring in the system?


End of Session 1

[email protected]

1





Objective

To understand, assess and increase university engagement with industry, the economy, society and community.

We must first understand the system that we are dealing with….

2


RESEARCH OUTPUTS

Publications

Processes

Materials

Technology

Know-how

Innovation

Skills

NewKnowledge

Researchers

Teaching

Networking/ Events

Consultancy

Professional

Development


Contract Research

Licensing

Company Creation

KE CHANNELS

Knowledge Flow

Jobs

New Products

New services

Turnover

Profit

R&D expenditure

% turnover from new products/ services

IMPACT

Start-up

Spin-out

Society

Government

Policy-makers

Entrepreneurs

Small Co’s

Big Co’s

USERS

Research

OTHER

FACTORS

OTHER

FACTORS


Studententerprise


Collaborative Research Licences Spin-outs

Public Good

Yes Yes ? No No No

Academic Reasons

? ? Yes Yes No No



3


Objectives

1. Do we have a view on what areas of activity we care about?

2. Do we know what the objectives are for each?

3. For those areas that we care about, do we know what success looks like?

Handout 1:

fill out the table for the spectrum of tech transfer activities


Possible Measures

1. For those areas that we care about, what could we count?

Activity:- What things could we count for each of these

Quality- What things could we count for each of these

Impact- What things could we count for each of these

4


Activity Measures

Upside

- Quantitative- Easy- Short term- Shows stuff is happening now

Downside

- Can be valueless/meaningless- Can be manipulated- Drives wrong behaviours


Quality Measures

Upside

- Measures quality – which is good.- Medium term and reasonable timescales

Downside

- By definition, qualitative- subjective- Very difficult to measure- Difficult to compare

5


Impact Measures

Upside

- Measures what is achieved- Demonstrates success (or not)

Downside

- Long term- Impact on who or what?- Contribution/attribution


Issues to Discuss

- What’s the right balance between activity/quality/impact

- Are the measures the same across the range of activities?

- What are some possible measures for each (HANDOUT 2)

6


Impact

- Impact is big in Australia (and elsewhere) just now

- It is viewed as the ultimate objective….

- What measures do you think can be applied?

- Impact Assessment Pilot exercise

….let’s look at some real examples


End of Session 2

Questions/Comments?

[email protected]

1





Objective



2


RESEARCH OUTPUTS

Publications

Processes

Materials

Technology

Know-how

Innovation

Skills

NewKnowledge

Researchers

Teaching

Networking/ Events

Consultancy

Professional

Development


Contract Research

Licensing

Company Creation

KE CHANNELS

Knowledge Flow

Jobs

New Products

New services

Turnover

Profit

R&D expenditure


IMPACT

Start-up

Spin-out

Society

Government

Policy-makers

Entrepreneurs

Small Co’s

Big Co’s

USERS

Research

OTHER

FACTORS

OTHER

FACTORS


So far we have….

- Considered approaches to measurement and assessment

- Looked at pros and cons of activity/quality/impact- Looked at impact assessment as an approach

Now, let’s consider $$$$

- Remember Handout 2 – middle row?- Is counting the $ an option?

3


Key Points

The ecosystem diagram shows the inter-relationship between research and impact. In particular, it shows the channels through which knowledge flows happen and shows the critical importance of research-users in creating impacts.

• The ERA has focussed on the left-hand side of the system, looking at research outputs – created by researchers with no need to involve research-users.

• The Impact Pilot is focussed on the right-hand end of the system, looking at the impacts – created by research-users with no need to involve researchers.

….neither of these really address the area where university tech transfer

happens….so let’s look at tech transfer activities.


There are a range of university/industry engagement activities.....

- Are they all tech transfer/commercialisation?- Which are most important to the University?- Which are most important for Australia?- Do they all make money?


4


Studententerprise



Public Good

Yes Yes ? No No No

Academic Reasons

? ? Yes Yes No No




The Returns from these Activities

Research

SMEs/Students

Public Good

Commercialisation

Knowledge Creation Making Money

5


The area that policy makers around the world care most about is innovation for Public Good.

-SME Engagement

-Innovative start-ups

-Entrepreneurship$ Profit

Public Good

Money Losing Activity


But the policy/support environment is based on the commercial model.

$ Profit

Making Money

6


From the University Perspective

- At the left we are agents of economic development, seeking to maximise public good – society and economy are the beneficiaries

- At the right we are venturers, seeking to maximise financial returns to the University – University is the beneficiary

- In the middle, we are co-creators of knowledge – company and university both benefit.

- These are really very different, therefore metrics are a challenge



At the left hand end, success results in losing money, whereas at the right end success results in making money. In the middle breakeven is success.THEREFORE: Profit isn’t a good measure

At the left end success usually looks like a very large number of small engagements, at the right hand end, success is usually from a very small number of big deals.THEREFORE: numbers of engagements isn’t a good measure

What about revenue?.....

7


The University Revenue Curve actually looks like…

$ Revenue



Research spectrum – financial picture

$M

$10M

$20M

$30M

$40M

$50M

$60M

Outreach R&D Contract Licensing Spin off2011 2010 2009

Example:UNSW data

8


The revenue-based model would suggest that:

− Public good and commercialisation activities are relatively small.

− The place where engagement happens between the research-base and

the user-base tends to be in collaboration/contracts.

− This is the biggest area and probably has the greatest potential for

growth….it’s the area that needs to be supported and incentivised in

order to get researchers to engage with users and users to engage with

researchers


Possible approach

− Revenue is a reasonable proxy for engagement across the spectrum

and are driven by knowledge flows through a range of channels and

mechanisms.

− Revenues also provide an objective, auditable, third party measure

of knowledge flows and the value to research-users.

9


Issues to Discuss- Engagement metrics are a practical approach to measuring

engagement with industrywhat are the downsides?should all categories be included?

- At the policy level, what particular types of engagement are preferred, if any?

- All of the current support is focused on the right hand end of the spectrum – is that what we want?

- Should there be modulation of funding to support/incentiviseparticular activities?


Thank you for participating

[email protected]@nsinnovations.com.au

3/10/2013

1


Measuring and improving the translation of research into social benefits


Objective



3/10/2013

2

Knowledge Exchange Model

RESEARCH OUTPUTS

Publications

Processes

Materials

Technology

Know-how

Innovation

Skills

NewKnowledge

Researchers

Teaching

Networking/ Events

Consultancy

Professional

Development


Contract Research

Licensing

Company Creation

KE CHANNELS

Knowledge Flow

Jobs

New Products

New services

Turnover

Profit

R&D expenditure


IMPACT

Start-up

Spin-out

Society

Government

Policy-makers

Entrepreneurs

Small Co’s

Big Co’s

USERS

Research

OTHER

FACTORS

OTHER

FACTORS

Key Points

The ecosystem diagram shows the inter-relationship between research and impact. In particular, it shows the channels through which knowledge flows happen and shows the critical importance of research-users in creating impacts.

• The ERA has focussed on the left-hand side of the system, looking at research outputs – created by researchers with no need to involve research-users.

• The Impact Pilot is focussed on the right-hand end of the system, looking at the impacts – created by research-users with no need to involve researchers.

….neither of these really address the area where university tech

transfer happens….so let’s look at tech transfer activities.

3/10/2013

3

There are a range of university/industry engagement activities.....

- Are they all tech transfer/commercialisation?- Which are most important to the University?- Which are most important for Australia?- Do they all make money?


Studententerprise



Public Good

Yes Yes ? No No No

Academic Reasons

? ? Yes Yes No No



3/10/2013

4

The Returns from these Activities

Research

SMEs/Students

Public Good

Commercialisation

Knowledge Creation Making Money

The area that policy makers around the world

care most about is innovation for Public Good.

-SME Engagement

-Innovative start-ups

-Entrepreneurship

$ Profit

Public Good

Money Losing Activity

3/10/2013

5

But the policy/support environment is based on the commercial model.

$ Profit

Making Money

From the University Perspective- At the left we are agents of economic development,

seeking to maximise public good – society and economy are the beneficiaries.

- At the right we are venturers, seeking to maximise financial returns to the University – University is the beneficiary.

- In the middle, we are co-creators of knowledge –company and university both benefit.

- These are really very different, therefore metrics are a challenge.

3/10/2013

6


At the left hand end, success results in losing money, whereas at the right end success results in making money. In the middle breakeven is success.THEREFORE: Profit isn’t a good measure.

At the left end success usually looks like a very large number of small engagements, at the right hand end, success is usually from a very small number of big deals.THEREFORE: numbers of engagements isn’t a good measure.

What about revenue?.....

The University Revenue Curve actually looks like…

$ Revenue


3/10/2013

7

Research spectrum – financial picture

$M

$10M

$20M

$30M

$40M

$50M

$60M

Outreach R&D Contract Licensing Spin off2011 2010 2009 Example:

UNSW data for illustration

purposes only.

The revenue-based model would suggest that:

− Public good and commercialisation activities are relatively small.

− The place where engagement happens between the research-base and

the user-base tends to be in collaboration/contracts.

− This is the biggest area and probably has the greatest potential for

growth….it’s the area that needs to be supported and incentivised in

order to get researchers to engage with users and users to engage with

researchers.

3/10/2013

8

Possible approach

− Revenue is a reasonable proxy for engagement across the spectrum

and are driven by knowledge flows through a range of channels and

mechanisms.

− Revenues also provide an objective, auditable, third party measure

of knowledge flows and the value to research-users.

How does this work in Practice?

Grant Cooley

3/10/2013

9

Issues to Discuss

- Engagement metrics are a practical approach to measuring engagement with industry

- what are the downsides?- should all categories be included?

- At the policy level, what particular types of engagement are preferred, if any?

- All of the current support is focussed on the right hand end of the spectrum – is that what we want?

- Should there be modulation of funding to support/incentiviseparticular activities?

1

NewSouth InnovationsUniversity of New South Wales

Engagement Metrics

@unswinnovators

2

1. Objectives for the Engagement Metrics model

2. The approach taken

3. Brief demonstration

4. Next steps

5. Questions / discussion

Outline

2

3

Objective

To establish an agreed set of metrics that many can use to enable improved understanding and comparability for collective benefit.

More specifically, to design a model that:

1. Builds on existing information sources

2. Provides useful and comparable information

3. Robust, consistent and verifiable data

4. Minimal administrative burden

4

The approach

Essentially a choice between two main approaches:

1. Annual report - then detail down to project level information

or

2. Build up from detailed sources like HERDC and project ledgers

Approach 1 was taken.

3

5

Research Outputs

OUTREACHBenefits

to society

OUTCOMESpin-out formation

Technology &

Research Developme

nt

Externally

funded contract activity

Licensing

UniversitiesExternal External

UNIVERSITY REVENUES

6

Definitions, inclusions and exclusions....Metric Description / Example

Outreach Philanthropic revenueIn-kind donations

Donations specifically in support of tech transfer

Technology Research and development

Linkage projects /Industry collaboration

External research Non government research revenue

Cat 3 research income

Royalties/License Revenue from licensed IPwhich has flowed from research activities

Assignment incomeRoyalty revenues

Spin-off Dividends received from spin off companies

The measurement of metrics would relate to a given period of time, for example, annual period.

4

7

Materials developed to date...

1. Explanatory guide to assist each institution categorising revenues sources (or excluding) from the Engagement Metrics model.

2. Spreadsheet model to capture relevant Engagement Metrics information

8

Preview of the spreadsheet model

Revenue from continuing operationsPart Included/

Excluded Outreach

Technology and


Externally funded Contract Research Licensing

Company Formation Excluded Total

$'000 $'000 $'000 $'000 $'000 $'000 $'000Australian Government Grants 1 ‐ 78,082 53,066 ‐ ‐ 567,077 698,225 HELP Australian Government Payment 1 ‐ ‐ ‐ ‐ ‐ 154,313 154,313 State and Local Government financial assistance 2 ‐ 27,594 ‐ ‐ ‐ 1,078 28,672 HECS ‐ HELP Student Payments Excluded 32,077 32,077 Fees and Charges Excluded 488,442 488,442 Investment Revenue 3 ‐ ‐ ‐ ‐ ‐ 39,937 39,937 Royalties, trademarks and licences Part Included 7,074 7,074 Consultancy and contracts 4 ‐ ‐ 51,557 ‐ ‐ 6,071 57,628 Other revenue 5 ‐ 5,122 ‐ ‐ ‐ 15,342 20,464

Gains on disposal of assets Excluded 3,256 3,256 Investment accounted for using the equity method Part Included 190 190 Other Income 5 ‐ 3,870 ‐ ‐ ‐ 13,530 17,400

Total Revenue and Income from Continuing ‐ 114,668 104,623 7,074 190 1,321,123 1,547,678

5

9

Sub-level analysis



Technology and


Externally funded Contract Research Licensing

Company Formation Excluded Total Audited

$'000 $'000 $'000 $'000 $'000 $'000 $'000 $'000Australian Research Council (ARC)i) Discovery

1.25 Project Part Included 33,641 33,641 33,641 1.26 Fellowships Part Included 17,805 17,805 17,805 1.27 Indigenous Researchers Development Part Included 222 225 447 447

Total Discovery ‐ 51,668 ‐ ‐ ‐ 225 51,893 51,893 ok

ii) Linkages1.28 Infrastructure Part Included 3,218 3,218 3,218 1.29 International Part Included ‐ ‐ 1.30 Projects Part Included 14,493 14,493 14,493

Total Linkages ‐ 17,711 ‐ ‐ ‐ ‐ 17,711 17,711 ok

iii) Networks and Centres1.31 Research Networks Part Included ‐ ‐ 1.32 Centres Part Included 2,964 2,964 2,964

Total Networks and Centres ‐ ‐ ‐ ‐ ‐ 2,964 2,964 2,964 ok

Total Australian Research Council ‐ 69,379 ‐ ‐ ‐ 3,189 72,568 72,568 ok

10

Annual Report Support schedules

GL Account Code 2012$'000

(f) Australian Research Council (ARC) Source

i) Discovery

Discovery Projects 0120 33,641

Discovery Fellowships 0135 17,805

Discovery Indigeneous Researchers Development 0144 447

Total Discovery 51,893

ii) Linkages

Linkages Infrastructure 0138 3,218

Linkages International 0137 ‐

Linkages Projects 0123 14,493

Total Linkages 17,711

iii) Networks and Centres

Centres 0112 2,964

Total Networks and Centres 2,964

Total Australian Research Council 72,568

6

11

Project level details…Project Description 0110 0120 0123 0134 0144 0710 0824 0826

Rsch-DIISR Instnl Grants

Schem

Rsch-ARC Discovery Projects

Rsch-ARC Linkage Projects

Rsch-State Govt Assist

Rsch-ARC Discovery Indigenou

s

Rsch-CRC's

Rsch-Health & Ageing

Rsch-Commonw

ealth Govt Assist

PS18720 Collab'n Childrens Hosp Westme 0.00 0.00 0.00 240,725.30 0.00 0.00 0.00 0.00

PS19815 CO2 Capture & Storage Misc 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

PS20432 Independent Monitoring of Brea 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

PS20721 ASSEMBLING THE TREE OF LIFE 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1,400.00

PS20970 Holly Seale Influenza work 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

PS21066 Aus Wetlands & River Ctr NCSRF 0.00 0.00 0.00 5,411.00 0.00 0.00 0.00 0.00

PS21247 OVERHEAD LEVY AMOUNT 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

PS22997 CMBB General Research Account 0.00 0.00 0.00 0.00 0.00 9,831.00 0.00 0.00

PS24883 ART/ ANZARD Report/ Feedback 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

PS24901 Agency4Clinical Innov NSW Heal 0.00 0.00 0.00 46,255.00 0.00 0.00 0.00 0.00

PS25484 OES Consult Service to Govt 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

PS25859 Staff Training - Govt provided 0.00 0.00 0.00 0.00 0.00 0.00 0.00 10,500.00

PS25900 Research Surplus-GlobalHealth 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

PS26367 Forensic Psychology Research 0.00 0.00 0.00 0.00 0.00 0.00 0.00 55,000.00

PS26500 Research surplus Funds 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

PS26609 LE120100036_Munroe, Paul R 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

PS26617 LE120100038_Wilkins, Marc 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

PS26851 CEEM Operating 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

PS26975 Paediatric Epilepsy 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

PS28349 Letnic Misc Research 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

PS28619 Juliet Richters Activities 0.00 0.00 0.00 2,000.00 0.00 0.00 0.00 0.00

PS28729 Activities_SHORT, Alison 0.00 0.00 0.00 2,000.00 0.00 0.00 0.00 0.00

12

Indicative completion timesYear 1:

1-2 days to read and understand guidance materials1-2 days to become familiar with the spreadsheet model2-3 days to source information 1-2 days to review and finaliseTotal of 4-9 days

Year 2 onwards:

3-4 days from start to end.

7

13

Questions ?

engagement metrics - unsw innovations...engagement metrics concentrate on the university activities...

Documents