delving deeper into the graduate experience

David Carroll, Research Associate

National survey of higher education graduates conducted since 1972

Current iteration is known as the Australian Graduate Survey (AGS)

Comprises:◦ Graduate Destinations Survey (GDS)◦ Course Experience Questionnaire (CEQ) or Postgraduate

Research Experience Questionnaire (PREQ)

Graduates surveyed approximately four months after the completion of their studies

Total of 119,447 responses received in 2008 (56.4 per cent response rate)

Key findings disseminated through a series of annual research reports produced by GCA

Examination of work and study activities of new graduates

Current AGS and time series

Compares relative proportions of graduates in discrete categories (e.g. % in FTE vs. % seeking FTE)

Postgraduate Destinations report also produced

Examination of earnings of recent higher education graduates

Current AGS and time series

Median starting salaries of bachelor degree graduates, along with comparative salary figures from other groups

Widely used for strategic decision making

Reports focus on descriptive statistics

Diverse readership meant keeping reports as ‘accessible’ as possible

Concern that additional analyses would limit their use as communication tools

How best to incorporate new analyses into these reports to complement and enhance the analyses contained within…

Without reducing their usefulness as non-technical communication tools

Graduate Salaries 2008 selected as a test case for the addition of new analyses◦ Do they provide additional value?◦ Possible within already tight timelines?

Three criteria for new analyses:◦ Methodologically appropriate◦ Complement and enhance◦ Easily understood and interpreted

Adjustment for Inflation

Statistical Significance Testing

Inflation refers to the sustained increase in the overall price level in an economy

Inflation is caused by:◦ Growth in the money supply (long term)◦ Supply and demand pressures (short term)

Important to consider impact of inflation when examining values of money over time

Necessary to consider the real value of median starting salary to understand the practical implications for graduates.

Calculated based on annual percentage change in the Consumer Price Index (CPI)

Dollars base = Dollars data (Annual CPI base / Annual CPI data)

This approach was chosen because:

◦ Familiar concept

◦ Widely accepted methodology

◦ Publicly available indices

Emphasis in these reports placed on differences between groups of interest

Descriptive statistics (of AGS sample)

Can between group differences be inferred to exist in the overall graduate population?

Significance tests incorporated into Graduate Salaries 2008 to address this question

Non-normal distribution confirmed by graphical and statistical examination

Selected tests needed to fit within existing report structure to ensure continuity

Results (and their interpretation) needed to be meaningful to a wide audience

Mann-Whitney U (Wilcoxon Rank-Sum)

Kruskal-Wallis

Differences flagged as:◦ Significant at the 5 per cent level (*)◦ Significant at the 1 per cent level (**)◦ Not statistically significant (n.s.)

All p-values provided in an appendix

Results of these tests discussed in the report in practical terms:

The largest difference in median salary by gender was in the field of optometry ($8,000), which was statistically significant at the 5 per cent level. This basically means that there is less than a one-in-twenty chance that this gender difference does not occur in the overall graduate population.

(Adapted from Graduate Salaries 2008)

Statistical significance

vs. practical significance

State/Territory of employmentMales

($, ‘000)Females($, ‘000)

Apparent difference in

GSS ($)

Diff. in GSS by gender

NSW 50.0 45.0 $5,000

WA 50.0 45.0 $5,000

Qld 46.0 44.0 $2,000

Vic. 45.8 44.0 $1,800

Tas. 44.1 43.1 $1,000

ACT 48.0 47.1 $900

NT 50.0 49.9 $100

SA 45.0 45.0 $0

State/Territory of employmentMales

($, ‘000)Females($, ‘000)

Apparent difference in

GSS ($)

Diff. in GSS by gender

NSW 50.0 45.0 $5,000 **

WA 50.0 45.0 $5,000 **

Qld 46.0 44.0 $2,000 **

Vic. 45.8 44.0 $1,800 **

Tas. 44.1 43.1 $1,000 n.s.

ACT 48.0 47.1 $900 n.s.

NT 50.0 49.9 $100 n.s.

Field of educationDiff. in GSS

by gender

Apparentdifferencein GSS ($)

Field of educationDiff. in GSS

by gender

Apparentdifferencein GSS ($)

Earth Sciences† $10,600 Biological Sciences $1,600

Optometry $8,000 Mathematics $1,500

Dentistry $6,000 Social Sciences $1,500

Arch. & Building $5,000 Veterinary Science $1,300

Art & Design $4,900 Agricultural Science $1,000

Economics, Business $4,000 Education $1,000

Medicine $3,000 Engineering† $1,000

Social Work† $2,400 Law $1,000

Computer Science $2,000 Physical Sciences $900

Humanities $2,000 Pharmacy $800

Paramedical Studies $2,000 Accounting $0

Psychology $1,900

† Female median GSS larger than male median GSS.

Field of educationDiff. in GSS

by gender

Apparentdifferencein GSS ($)

Field of educationDiff. in GSS

by gender

Apparentdifferencein GSS ($)

Earth Sciences† n.s. $10,600 Biological Sciences n.s. $1,600

Optometry * $8,000 Mathematics n.s. $1,500

Dentistry n.s. $6,000 Social Sciences n.s. $1,500

Arch. & Building ** $5,000 Veterinary Science n.s. $1,300

Art & Design * $4,900 Agricultural Science n.s. $1,000

Economics, Business ** $4,000 Education * $1,000

Medicine * $3,000 Engineering† n.s. $1,000

Social Work† n.s. $2,400 Law n.s. $1,000

Computer Science n.s. $2,000 Physical Sciences n.s. $900

Humanities ** $2,000 Pharmacy n.s. $800

Paramedical Studies ** $2,000

Psychology n.s. $1,900

† Female median GSS larger than male median GSS.

Field of educationDiff. in GSS by industry†

Apparentdifferencein GSS ($)

Field of educationDiff. in GSS by industry†

Apparentdifferencein GSS ($)

Pharmacy $13,500 Economics, Business $7,000

Education $13,000 Social Sciences $7,000

Art & Design $11,100 Computer Science $6,500

Optometry $10,000 Paramedical Studies $6,500

Humanities $9,500 Agricultural Science $5,500

Accounting $9,400 Mathematics $5,500

Psychology $9,000 Biological Sciences $5,300

Arch. & Building $8,900 Earth Sciences $5,000

Engineering $8,900 Law $5,000

Dentistry $8,500 Medicine $3,000

Physical Sciences $7,500 Veterinary Science $0

Social Work $7,500

† Aust. Govt, State Govt, Public Health, Prof. Pract. Ind./Com., Schools, Tert. Ed.

Field of educationDiff. in GSS by industry†

Apparentdifferencein GSS ($)

Field of educationDiff. in GSS by industry†

Apparentdifferencein GSS ($)

Pharmacy ** $13,500 Economics, Business ** $7,000

Education ** $13,000 Social Sciences * $7,000

Art & Design ** $11,100 Computer Science n.s. $6,500

Optometry n.s. $10,000 Paramedical Studies ** $6,500

Humanities ** $9,500 Agricultural Science n.s. $5,500

Accounting ** $9,400 Mathematics n.s. $5,500

Psychology ** $9,000 Biological Sciences ** $5,300

Arch. & Building ** $8,900 Earth Sciences n.s. $5,000

Engineering ** $8,900 Law n.s. $5,000

Dentistry n.s. $8,500 Medicine ** $3,000

Physical Sciences n.s. $7,500

Social Work ** $7,500

† Aust. Govt, State Govt, Public Health, Prof. Pract. Ind./Com., Schools, Tert. Ed.

Graduate Destinations focus on relative proportions of graduates within categories

Two proportion z-test proposed to examine two subgroups of interest:◦ Proportion of graduates engaged in particular activities◦ Proportion engaged in particular activities in different

years

Graduates in full-time employment

p1 p2 Sig. z-score

Bachelor Honours * Bachelor Pass 85.5 85.1

Bachelor Grad Entry * Bachelor Pass 85.5 85.1

Masters Coursework * Masters Research 88.7 87.6

Masters Coursework * G/PG Diploma 88.7 89.0

Graduate Certificate * G/PG Diploma 95.1 89.0

2008 * 2007 85.2 84.5

Males * Females 85.5 85.0

Graduates in full-time employment

p1 p2 Sig. z-score

Bachelor Honours * Bachelor Pass 85.5 85.1 0.484 n.s. 0.70

Bachelor Grad Entry * Bachelor Pass 85.5 85.1 0.516 n.s. 0.65

Masters Coursework * Masters Research 88.7 87.6 0.503 n.s. 0.67

Masters Coursework * G/PG Diploma 88.7 89.0 0.562 n.s. 0.58

Graduate Certificate * G/PG Diploma 95.1 89.0 0.000 ** 11.04

2008 * 2007 85.2 84.5 0.004 ** 2.87

Males * Females 85.5 85.0 0.156 n.s. 1.42

Kruskal-Wallis Test indicates that at least one pair of groups are different

No multiple comparison tests for Kruskal-Wallis included in Graduate Salaries 2008◦ Recommended that readers compare medians to identify

extreme groups (after Dytham, 2003)

Considering adding multiple comparison tests in Graduate Salaries 2009

Multiple comparison procedure for Kruskal-Wallis when sample sizes are not equal

Minimises Type I errors (falsely rejecting H0) when performing multiple pairwise comparisons (Q statistic)

Field of educationDiff. in GSS by industry†

Apparentdifferencein GSS ($)

Field of educationDiff. in GSS by industry†

Apparentdifferencein GSS ($)

Pharmacy ** $13,500 Economics, Business ** $7,000

Education ** $13,000 Social Sciences * $7,000

Art & Design ** $11,100 Computer Science n.s. $6,500

Optometry n.s. $10,000 Paramedical Studies ** $6,500

Humanities ** $9,500 Agricultural Science n.s. $5,500

Accounting ** $9,400 Mathematics n.s. $5,500

Psychology ** $9,000 Biological Sciences ** $5,300

Arch. & Building ** $8,900 Earth Sciences n.s. $5,000

Engineering ** $8,900 Law n.s. $5,000

Dentistry n.s. $8,500 Medicine ** $3,000

Physical Sciences n.s. $7,500

Social Work ** $7,500

† Aust. Govt, State Govt, Public Health, Prof. Pract. Ind./Com., Schools, Tert. Ed.

Field of educationDiff. in GSS by industry†

Apparentdifferencein GSS ($)

Field of educationDiff. in GSS by industry†

Apparentdifferencein GSS ($)

Pharmacy ** $13,500 Economics, Business ** $7,000

Education ** $13,000 Social Sciences * $7,000

Art & Design ** $11,100 Computer Science n.s. $6,500

Optometry n.s. $10,000 Paramedical Studies ** $6,500

Humanities ** $9,500 Agricultural Science n.s. $5,500

Accounting ** $9,400 Mathematics n.s. $5,500

Psychology ** $9,000 Biological Sciences ** $5,300

Arch. & Building ** $8,900 Earth Sciences n.s. $5,000

Engineering ** $8,900 Law n.s. $5,000

Dentistry n.s. $8,500 Medicine ** $3,000

Physical Sciences n.s. $7,500

Social Work ** $7,500

† Aust. Govt, State Govt, Public Health, Prof. Pract. Ind./Com., Schools, Tert. Ed.

Industry Group

RA

Aust. Govt 926.43

Prof. Pract. 892.54

Public Health 880.50

Tert. Ed. 844.82

State Govt 820.02

Schools 810.10

Ind./Com. 533.23

Q 533.23 810.1 820.02 844.82 880.5 892.54 926.43 RB

Ind. / Com.

SchoolsState Govt

Tert. Ed.Public Health

Prof. Pract.

Aust. Govt

Industry Group

Industry Group

RA

Aust. Govt 926.43    

Prof. Pract. 892.54     0.72

Public Health 880.50     0.11 0.41

Tert. Ed. 844.82     0.30 0.75 1.19

State Govt 820.02     0.35 0.54 1.48 1.92

Schools 810.10     0.19 0.52 0.63 1.87 2.28

Ind./Com. 533.23   7.57 6.78 5.32 3.27 11.87 9.89

Q 533.23 810.1 820.02 844.82 880.5 892.54 926.43 RB

Ind. / Com.

SchoolsState Govt

Tert. Ed.Public Health

Prof. Pract.

Aust. Govt

Industry Group

QCrit = 3.038 (p < .05; k = 7)

Industry Group

RA

Aust. Govt 926.43    

Prof. Pract. 892.54     0.72

Public Health 880.50     0.11 0.41

Tert. Ed. 844.82     0.30 0.75 1.19

State Govt 820.02     0.35 0.54 1.48 1.92

Schools 810.10     0.19 0.52 0.63 1.87 2.28

Ind./Com. 533.23   7.57 6.78 5.32 3.27 11.87 9.89

Q 533.23 810.1 820.02 844.82 880.5 892.54 926.43 RB

Ind. / Com.

SchoolsState Govt

Tert. Ed.Public Health

Prof. Pract.

Aust. Govt

Industry Group

QCrit = 3.038 (p < .05; k = 7)

Inflation adjustment permits salaries over time to be examined on a common base◦ Practical implications for graduates

Statistical significance tests allow for more reliable inferences to the overall population

Even more robust and comprehensive understanding of graduate outcomes

David CarrollGraduate Careers Australiaresearch@graduatecareers.com.au+61 3 9605 3700

www.graduatecareers.com.auwww.graduatecareers.com.au