1

Analytical Support for Decision

Making Assignment

Group Hashtag

201266929

201273815

201269197

201285376

201273669

2

Contents Introduction ............................................................................................................................................................ 3

The Proposition .................................................................................................................................................. 3

The Problem ....................................................................................................................................................... 3

The Proposal ....................................................................................................................................................... 3

PART A DATA ANALYSIS AND RECOMMENDATIONS ............................................................................................ 4

Executive Summary ................................................................................................................................................ 4

Data Cleansing & Preparation ................................................................................................................................ 6

Background ........................................................................................................................................................ 6

Reorganising the data set ................................................................................................................................... 6

Initial work on the data set ................................................................................................................................ 6

Data Selection (including variable selection) ..................................................................................................... 7

Generating additional data ................................................................................................................................ 8

Visual Presentation ............................................................................................................................................ 8

Analysis of Data ...................................................................................................................................................... 9

Exploratory Analysis using Summary Statistics .................................................................................................. 9

Advanced Analysis Inference & Regression ................................................................................................... 13

Recommendations for Head of Department and Minister of Education ........................................................... 15

Part B .................................................................................................................................................................... 17

Value Tree ........................................................................................................................................................ 17

Scoring .............................................................................................................................................................. 18

Analysis ................................................................................................................................................................. 18

Structuring the Problem ................................................................................................................................... 18

Criteria ......................................................................................................................................................... 18

Alternatives .................................................................................................................................................. 19

Uncertainties ................................................................................................................................................ 19

Stakeholders ................................................................................................................................................ 20

External / Environmental ............................................................................................................................. 20

Weighting ......................................................................................................................................................... 21

Tier 2 ............................................................................................................................................................ 21

Tier 1 ............................................................................................................................................................ 22

Outcomes ......................................................................................................................................................... 23

Weight Sensitivity Analyses .............................................................................................................................. 25

Recommendations ....................................................................................................................................... 26

References ............................................................................................................................................................ 28

Appendix.29

3

Introduction

The Proposition

We are a group of students working for the Department of Education in Indonesia and our Line

Manager, the Head of the Department, has asked us conduct a quantitative analysis of Indonesia

with regards to its educational performance. Once we have analysed the data we are to provide a

detailed report for our Manager, who will then use this report to brief the Minister of Education.

This report will provide necessary information to the Minister in order for him to make informed

decisions on the future of education in Indonesia.

The Problem

Whilst our department provides guidance to the Minister of Education on social and economic

issues, that may affect the success and delivery of Education in the country, we have been asked to

conduct our analysis over several broad areas, utilising the data from the World Bank. These include:

Comparison of Indonesia with surrounding countries in the region

Comparison of Indonesia with developed countries

Comparison through time

The Minister has also asked us to include exploratory and inferential analysis within our report in

order for him and his department to better understand the process of data management so that he

may use this model as the basis for all future analysis going forward.

The biggest challenge that we face is trying to use and analyse the data provided by the World Bank.

Much of the data is either incomplete or irrelevant. Through a process of data cleansing we will

provide meaningful and coherent analysis for the Minister in his decision making process for

education in Indonesia.

The Proposal

The report is split into two main sections to make the findings of the report more easily understood

by the Minister and our Head of Department.

The first section of the report looks at the process of data cleansing, explaining how we converted

the data from the World Bank into a more meaningful, manageable database and how we

established the variables to be considered and our preparation process for using the data. We then

discuss how we selected, used and interpreted the visual displays of data and our use of summary

statistics in our analysis. Finally we summarise our selection and interpretation of the data using

inference, regression, normal distribution and confidence intervals.

4

We finalise this section of the report by providing some detailed recommendations to the Minister in

order to help him draw some meaningful conclusions regarding education in Indonesia.

The second part of the report discusses the Ministers options for the future of the country and his

desire to increase the number of graduates with analytical skills in Indonesia. He firmly believes that

Indonesia should have postgraduates with strong core subjects and has identified three key options

that are open to him:

Option A Send students to study abroad

Option B Bring teaching into Indonesia

Option C Develop courses within Indonesia.

The chosen outcome of these options will clearly have major political, social and economic

implications and as such, the decision must be the correct one for the Minister and his department.

For this part of the report we used a Multi Criteria Decision Analysis software tool called VISA.

Through a series of stakeholder role play scenarios, lengthy discussions and several variations of

scenario mapping, we have provided suitable information to the Minister that should help him in his

decision for the future of postgraduates in the country. By using this software tool we were able to

demonstrate the potential risks, successes and wrong decisions in all scenarios and therefore the

Minister would be suitably informed of all the variable choices open to him and his department. We

complete this section of the report with our recommendations to the Minister based on the

scenarios and variables that we have demonstrated to him and his department.

All supporting data and evidence is attached in the Appendices and has been referenced throughout

the report.

PART A DATA ANALYSIS AND RECOMMENDATIONS

Executive Summary

The report provides the details of the analysis conducted for the Ministry of Education to provide an

insight into factors associated with the enhancement of education. The analysis was conducted by

comparing the selected indicators of Indonesia against an equal number of regional and developed

countries. In order to undertake a comprehensive review, data for countries and indicators was

utilised from the World Bank (The World Bank Group, 2013).

Indonesia was compared with the selected countries on identified indicators using the most

available current data. The techniques utilised for the analysis were:

Visual Analysis Bar Charts

Visual Analysis Trend Charts

Ranking Tables using a balanced scorecard approach

Normal Distribution

Regression

5

Visual analysis of bar charts was used to analyse the position of Indonesia compared to the selected

countries, while trend charts showed the data pattern of the indicator. The indicators used for the

analysis could not completely facilitate the use of ranking tables. Some of the indicators provided

were disproportionate for comparison. However valuable information was gained on attempting to

analyse the ranking table. The use of normal distribution allowed certain assumptions to be made

regarding the current standing of Indonesia with respect to the overall population for a key

indicator. Linear regression analysis was performed to determine the level of relationship between

two educational indicators.

Indonesia was ranked top among the regional countries with respect to primary completion rate and

had caught up with the developed countries. We would suggest this is due to the government policy

of compulsory education for the first nine years. The focus on spending on education is however in

substantial decline. Spending policy does not follow a pattern in regional countries while the

developed countries have a pattern. Highly volatile trends for some indicators appear to

demonstrate a lack of long term strategy from the Indonesian government.

Indonesia's public spending on education as % of GDP is in the bottom quartile of the world

population requiring the Ministry of Education to review this figure and make strategies to improve

this. Although Indonesia has maintained a pupil teacher ratio competitive to developed countries

there is a requirement to introduce advanced training and teaching programmes into education.

The number of students attending secondary education is in decline. Qualitative research will

provide insights into the reasons behind the decline in the number of students attending secondary

education despite the fact this being mandatory for the first three years (AngloINFO, 2013).

It is critical for the analysis to be conducted on current, consistent and transparent data on

indicators directly related and supporting education. The report is limited to quantitative analysis.

Qualitative analysis on the indicators could provide an insight into the reasons for the trends

observed and complement the recommendation.

It is worth noting that data for certain indicators could be disputed. For instance, some percentage

figures were over 100%, even though based on the indicator description, this would be

mathematically impossible. As a result, it was important to constantly question the data and

conduct the analysis with a level of scepticism. Therefore, it is advised that the recommendations

made are also treated with care.

6

Data Cleansing & Preparation

Background Data cleansing is a process of reviewing data in its raw form and identifying, correcting and removing corrupt or inaccurate data. Inconsistencies and anomalies may be the result of incorrect data being reported, data not being available for certain countries or may be due to corruption in the transfer of data into the database. A significant amount of data was provided and this had to first be configured into a manageable and meaningful format, before it could be accurately grouped and analysed. Therefore, the first objective of the process was to reconfigure and present the data in a consistent and user friendly manner to allow analysis. The raw data was supplied for a number of countries in the East Asia region for a variety of different indicators.

Reorganising the data set The initial data set was in a format that was difficult to manage. To overcome this, data was sorted into filter fields to create the ability to select specific data samples which could be easily compared and measured against. The data was then sorted, which enabled the grouping of key indicators into alphabetical order. This brought together the data from each country, for each key indicator, into one block, making it easier to compare. This was further enhanced by adding an empty row between each set of indicators, which helped to visually highlight each section. Finally, the number format of the data was changed to two decimal places to make the figures more manageable. The results of this process can be seen in the Excel Appendix on the Orange worksheet (Hashtag, 2013).

Initial work on the data set When we first looked at the data set, we recognised that:

- A number of the key indicators did not seem relevant to the task - A number of countries had no data against a large number of the indicators, especially

between 1991 and 2000 - The only data provided for a developed country outside of the region was for the UK

By referring to the World Development Bank (WDB) data in Google Visualization we were able to identify additional indicators that were relevant to education. Furthermore, by using the WDB map and through trial and error of various analogies between Indonesia and several countries against different educational indicators, we identified a number of developed countries across the world, to provide a comparison point for Indonesia. This gave us 14 countries (regional and developed) and when added to our initial dataset, 37 possible indicators to work with. The new data was put into the same format as the original dataset and the years 1961-1990 removed to make both sets of data comparable. In the original data set, any seemingly unrelated indicators were removed. Finally, both sets of cleansed data were combined in a new spread sheet and separated into sections (Developed and Region). Indonesia was copied into the top cell of each section to allow for easy comparison in visual presentation. This cleansed data can be seen in the Excel Appendix on the Red worksheets.

7

Data Selection (including variable selection)

It was recognised that 37 indicators was too many to be used to provide manageable amounts of data for analysis and so this needed to be reduced. This was done by relevance of the indicator to this specific task and by the availability of data, across a number of countries. Four categories were established to help narrow down the indicators. These were:

- Spending on education - Areas of spend e.g. teachers - Measurable performance outcomes - Alternatives to public education

Through this process, the number of indicators was reduced to 17. The next step involved cross checking the available data for these indicators across each country.

There was a very low availability of consistent data for 1991-2000 and as it took time for the data to be gathered and reported, there was also minimal data from 2011-2012. The majority of the data set was most complete from 2000-2010 and so for the sake of comparison; we agreed to focus on this time frame. However, there were still indicators that had insufficient data attached to them. For example, data for levels of adult literacy would have been very useful for analysis but the data was sporadic at best across all countries and not reported at all for Indonesia. For this reason, we were unable to utilise it as a key indicator. By applying the same rationale to all indicators, we were able to further reduce the total number of key indicators to 13.

Figures 1 & 2 below summarise the final regional & developed countries chosen as well as the list of indicators which we felt were most relevant and had the best quality of data:

Whilst we wanted to compare the most current data, the gaps in the data set made this difficult. To overcome this we decided to select the most up to date, most complete data for each key indicator, from the data set. For example, when considering the indicator Primary completion rate, total (% of relevant age group) the year 2010 was selected as this column provided the most data to work with and therefore would create the most meaningful comparison for analysis.

Each key indicator was given a separate tab in the excel file and the data for all countries (categorised into regional and developed) was copied into each sheet. This created a workbook with the data organised and categorised into relevant, easy to read sections.

Regional Comparison Countries

Developed Comparison Countries

China Finland

Hong Kong, SAR Germany

Japan New Zealand

Malaysia United Kingdom

Philippines United States

Singapore

Indicator

Expenditure per student primary education % per capita GDP

Expenditure per student secondary education % per capita GDP

Expenditure per student tertiary education % per capita GDP

Public Spending on Education % GDP

Public Spending on Education % Government Expenditure

Primary Completion rate (%)

Primary Teacher numbers

Secondary Education Teacher numbers

Secondary Education Student numbers

Pupil Teacher Ratio primary education

Pupil Teacher Ratio secondary education

Progression to Secondary Education

Secondary Education Vocational Student numbers

Figure 1 Chosen comparison countries against

Indonesia

Figure 2 Chosen Educational Indicators

8

Generating additional data Using this newly created data set, a number of additional calculations were completed to support the analysis. This included finding values for the:

- Mean and median for each country (per key indicator). We ultimately decided that the mean would be the most useful and calculated this to give us an indication of the average.

- Standard deviation (to ascertain the variance of the data from the mean). - Ranking of countries. This was a useful way of ordering each country in terms of current

position (by key indicator) to compare against other countries. This was done for each group of developing and regional countries and for the sample as a whole.

This process involved a level of trial and error; with some calculations not providing meaningful results depending on the quality of data in the corresponding data set and where size of a country meant it was not easily comparable with others. The creation of graphs and charts was essential for analysing the data and helped with the identification of trends and issues.

Visual Presentation The visual presentation of data served two purposes; firstly, to aid the analysis and secondly, to allow clear communication of the findings, highlighting key points and aiding understanding of those being informed. By generating visuals of the data, it was easier to check for accuracy. For example, when viewing a series of graphs relating to expenditure per student, it became evident that one country was significantly greater than any other. This was cross checked with the data set and although the values matched, it was recognized that inaccurate reporting may have occurred. In terms of the analysis, using the interactive charts from Google Visualization was very beneficial for quickly bringing data to life, especially over a large time period. This helped us to select which areas to focus on for analysis. Through the creation of graphs with two Y-axes, it was possible to visibly compare, understand and analyse data. Utilizing tools on Excel, it was also possible to predict trends and possible convergence points in the future.

Due to the number of countries and years initially identified, the majority of the graphs generated presented too much data and so were incoherent and difficult to compare and to analyse. To overcome this, we reduced the number of countries being compared and adjusted the timeline to increase the amount of consistent data available per country.

Through the analysis of the data set, a significant number of graphs, charts and tables were generated, both as snap shots of the position in a particular year and as a trend over time. This was very helpful for analysis, but it was recognized that to include this level of information in a report would be confusing for the reader and that it may dilute the message being conveyed. Whilst all data was relevant, the most important data (positive and negative) was added to the main body of the report, with supporting information added to the Excel appendix. There were a number of other charts that were discarded altogether, despite the time invested in generating them. The lack of available data in some areas had a negative effect on some of the charts, causing confusion when looking at them. For aesthetic reasons, it was decided to remove the corresponding countries from the charts/ graphs however; these were retained in the data sets as evidence that they had not been

9

deliberately omitted to create a false picture. As a final aid to those reading the report, Indonesia was represented by the colour black in all charts/ graphs, to avoid confusion.

Analysis of Data

Once the data had been properly cleansed to the countries and indicators that we were interested

in, it was now appropriate to analyse the data in two ways:

Firstly, to compare and contrast the indicators at both a regional level (i.e. looking at how

Indonesia compared with other countries in the surrounding region) and on a global scale

(specifically how Indonesia compared with developed countries around the world). This

would involve the identification of trends between these countries and how they varied over

time. It would also involve the comparison of countries in a particular year to see how they

performed.

Secondly to use a variety of statistical tools. This consisted of inferential analysis to see if

assumptions could be made regarding Indonesias performance in relation to the total

population and specifically, trying to understand if any of the data could be applied to a

normal distribution to make assumptions regarding its performance relative to the overall

population. Finally, regression analysis was undertaken to understand if there were linear

relationships present between any two indicators for Indonesia and therefore if we could

make assumptions on what would drive improvements in any particular indicators.

Successful analysis of both of the above would enable us to make recommendations to the Head of

Department and the Minister of Education for recommended improvements.

Exploratory Analysis using Summary Statistics

The first part of the analytical process involved exploratory analysis of the data to present key

findings in relation to Indonesias performance in the key educational indicators versus regional and

developed countries.

Comparison of Regional & Developed Countries

The best way to compare Indonesias performance with other countries over time was using line

charts to view trends. In order to compare performance across countries at a particular point in

time, bar charts were created. These were ranked from high to low so that Indonesias rank within

the sample could be clearly identified.

The detailed results of this analysis for each of these indicators can be found in the Excel Workbook

with each of the relevant tabs highlighted in Green. For the purpose of this report, we selected 6

key indicators which would give the Minister of Education a balanced view of Indonesias

comparative performance by choosing 2 indicators where Indonesia appeared to perform better

10

than its peers, 3 indicators which indicated areas of concern and 1 indicator which on the surface

showed signs of positivity with an underlying concerning trend. These are summarised below.

Indicators showing positive trends for Indonesia versus peer countries

Indicator 1 Primary completion rate

The bar chart below shows Indonesia in a positive light for the indicator Primary Completion Rate.

Based on the 2009 World Bank data, it has the highest ranked value among both the surrounding

regional countries and the developed countries. This can be seen in Figure 3 below:

The trend charts in the Excel Workbook also show quite a sharp upward trend relative to these

countries and from this we can assume that there has been a strong focus to improve results in the

primary education sector in Indonesia over the past 6 years. This is an important fact to consider in

that it also determines the long term demand for a good tertiary education system and employment

opportunities for this population in the long term.

Indicator 2 Pupil teacher ratio

When looking at this indicator, it was recognised that the large majority of countries, including

Indonesia, showed a downward trend over time. In the context of this indicator, this was viewed as

a positive result, as we would assume that a lower pupil teacher ratio would imply a better quality of

education (Figure 4 below).

When comparing this within the region for primary education, Indonesia performs well amongst its

surrounding countries, with only the Philippines being a noticeable outlier to the general sample. In

comparison to the developed countries, Indonesia still has some catching up to do but has made

some significant improvements, particularly during the period 2006 to 2010. Finally, one area in

which Indonesia sees particular progress is in its pupil teacher ratio for secondary education, which

shows this as being very strong against the developed countries at an average of 12.4 over the

period 2006 to 2010.

Figure 3 Primary Completion Rate comparison

Figure 4 Pupil Teacher Ratio comparison

11

Indicators showing challenging trends for Indonesia versus peer countries

Indicator 3 Expenditure per student

The first indicator identified as being one of potential concern was Expenditure per Student as a % of

GDP per capita. For primary and secondary education, Indonesia sits in a relatively low position

compared to the surrounding regional countries (albeit in a better position than the Philippines and

surprisingly, Singapore). It also performs very poorly against the developed countries and was the

worst in this sample. Perhaps more worryingly, there was a downward trend in this indicator

between 2009 and 2010 for both primary and secondary education which was in contrast to an

upwards trend for both regional and developed countries, though it is difficult to determine whether

or not this was only a blip as the data for 2011 and 2012 was not available.

Figure 5 summarises some of these observations below for primary education, though the full suite

of charts can be found in the Excel Workbook.

Indicator 4 Public Spending as a % GDP

In a similar vein to Indicator 3, Indonesia performs in a very similar fashion between both regional

countries (i.e. relatively low position) and developed countries (i.e. worst performer) for Public

Spending as a % GDP. This is not surprising as these two indicators are very closely linked in that

they relate to public spending and the GDP of each country. Figure 6 shows this comparison below:

Indicator 5 Progression to Secondary Education

The main thing that jumped out of this indicator was the sudden downward trend between 2006

and 2007 and that at the point of the most recent data for this indicator in 2009, this situation had

not improved. In comparison to both the regional and developed countries, Indonesia appears to

fare the worst in the sample set. However, it is important that with a very small level of data across

Figure 5 Expenditure per student primary education comparisons

Figure 6 Public spending as a % GDP comparisons

12

a limited timeframe and across few countries, the above conclusions cannot be taken as concrete

evidence of Indonesia performing poorly in this indicator, although from the data given, we can

assume enough to be concerned. These trends are shown in Figure 7 below:

Indicator 6 Public Spending as a % Government Expenditure

Finally, we identified Public Spending as a % of total Government Expenditure as an interesting trend

as this did show Indonesia as the highest ranked compared to the developed countries with a decent

performance amongst the regional countries. It was however interesting to note the significant

variation in the trend for Indonesia, in particular compared to the developed countries and this

posed questions as to how consistent the spending policy on education was or if it was treated as

more of a fire-fighting exercise. This could indicate a lack of a long term strategic plan in improving

Indonesias educational system as the spending levels are very sporadic (see Figure 8 below).

Ranking Tables

We also decided to create ranking tables across each of the indicators to effectively create a

balanced scorecard of Indonesias performance ranked against the other sample countries. Each

country was ranked within each indicator and within each sample of regional and developed

countries. An overall ranking was also created across the 2 sample groups. The ranking tables

associated with this analysis can be found in the Excel Workbook with the relevant tab highlighted in

Yellow.

Figure 8 Public Spending % Government Expenditure

Figure 7 Progression to Secondary Education % students

13

After considerable debate, it was decided not to draw any firm conclusions from this analysis, as

some of the indicators were directly related to the size of the country and therefore not a fair

representation of performance. For example, China automatically ranked in 1st place for indicators

relating to numbers of teachers and pupils, simply due to the size of the country. Nevertheless, for

those indicators that demonstrated a fair comparison of performance, it was still interesting to

analyse how Indonesia compared across both sets of countries. Notably:

Indonesia ranked in 1st place across both sets of countries for Primary Completion Rate

Indonesia also ranked in 1st place for enrolment into primary and secondary private

education, however it was important to be aware of what private education meant within

the context of each countrys educational system as this could be misleading

Advanced Analysis Inference & Regression

Inference using Normal Distributions

We decided to look at a selection of indicators and determine their distribution across the

population of World Bank data. Specifically, we were looking for any instances where a Normal

Distribution was present. This would enable us to draw assumptions on the data in terms of

Indonesias performance versus the overall population.

Five of the indicators were chosen to determine whether such a distribution was present. The

relevant sheets and charts relating to this are marked in the Excel Workbook with Blue tabs. The

location of Indonesia within each chart was highlighted in red to quickly see its position within the

overall population.

Following this analysis, it was clear that only 1 of the chosen indicators could be approximated to a

normal distribution, namely Public Spending on Education as a % GDP, as can be seen in Figure 9

below:

Using this assumption to the Normal Distribution, the following conclusions could be drawn:

Mean average = 4.77; Standard Deviation = 1.86;

This gave a z score for Indonesia = (2.90-4.77)/1.86 = -1.00

Figure 9 Normal Distribution of Public Spending on Education as a % GDP

14

Using the Normal Distribution table, the corresponding proportion of the population where

public spending as a % GDP was less than that of Indonesia = 15.87%

This effectively placed Indonesia in the bottom quartile of the data, with 84.13% of the

population of countries spending more money as a % of their GDP on education

Clearly this is a powerful message for the Head of Department and Minister of Education to

understand when reviewing educational policy and future spending plans.

Future Projections based on Past Analysis using Regression Techniques

Regression is a technique of determining a relationship between two indicators. Using the graphical

pattern of the indicators and significance they had on education, we identified seven pairs of

indicators that could potentially have an influence on each other. The analysis for each of these can

be found in the Excel workbook under the White tabs. The decision to use linear regression

provided us with an opportunity to determine if the data available on the indicators had a direct

relationship with each other.

We started the analysis by plotting the data in a scatter chart to visually look for trends. Visual

analysis was followed by creating a trend line using excel function and extrapolating the correlation

coefficient. We used the correlations co-efficient comparison with 2/ and the confidence interval

of the slope, not to contain a zero as a check to ensure it there is valuable information provided by

performing the regression. We also used the R-Squared value to eliminate pairs due to the data only

being able to explain a low proportion of variability.

From the seven pairs analysed, two pairs had a relationship where the variance was above 75%.

They were:

Secondary Education Student Numbers and Secondary Education Teacher Numbers

Public Spending % GDP and Public Spending % Govt. Expenditure

Figure 10 shows a clear relation between number of students (which we used as the independent

variable) and teacher numbers (the dependent variable).

Figure 10 Linear Regression relationship between Secondary Education Student Numbers and Secondary

Education Teacher Numbers

15

Disappointingly, we could not find a clear relationship between Public Spending on Education and

Primary Completion Rate or between Expenditure per Student in Primary Education versus Primary

Completion rate. This would make it more difficult to find a link between public spending and

educational performance.

However, for those indicators where a linear relationship was found, the regression modelling

provided us with an opportunity of looking at future predictions with confidence intervals. Again,

the details of this can be found in the Excel Workbook for the regression analysis (White tabs). This

provided us with an opportunity to predict the course of the indicator and make appropriate

recommendations to the minister.

Recommendations for Head of Department and Minister of Education

Following analysis of the available data, we would recommend four priority areas for further

consideration and appropriate action:

1. Figure 6 demonstrates that Public Spending on Education as a percentage of GDP is consistently lower than many countries in the East Asia region and significantly lower than the developed countries. Furthermore, as outlined in Figure 9, only 16% of worldwide countries tracked by the WDB, report spending less on Education as a percentage of GDP than Indonesia. Over time, it is likely that this underinvestment will weaken the Education framework. To prevent this, the Department of Education must review and update the long term investment strategy for Education. This could be aided by comparing current spend plans to those of developed countries. The Department should also consult with industry to ensure that future employment needs are understood and addressed through future spend plans.

2. As demonstrated by Figure 10, a relationship exists between the number of teachers and pupils. Figure 4 highlights that the pupil teacher ratio has been reducing over the last 10 years, suggesting that class sizes are shrinking, which may increase the quality of learning. The Department of Education should consider the impact of an increasing population and an ageing teaching workforce on this ratio and ensure that a robust teacher training programme is in place, to maintain these class sizes. As new teaching methods evolve, it may be that there is a desire to decrease this ratio further. Through on-going investment in teacher training, Indonesia will be able to maintain and grow the number of qualified teachers.

3. From the Excel Workbook it can be seen that whilst there are comparably high numbers of young people completing primary education, there is a dip in the numbers going onto secondary school. As attendance at secondary school is mandatory for the first three years, it would be expected that the numbers accessing secondary would mirror that of the primary completion level. Some additional qualitative research is necessary to establish why this is the case and depending on the findings, action may need to be taken to encourage parents to send their children to secondary education in Indonesia (as opposed to going abroad for example) and to enforce the compulsory attendance of those young people who find unlawful pathways into work. Increasing the number of young people attending secondary school should have a positive impact on both the level of education in the adult population and the number of young people going onto tertiary education.

16

4. In order to accurately compare Education in Indonesia with other countries and identify trends, good practice and areas for concern, data must be captured and reported more consistently. Whilst there is some very useful data, there were a number of additional indicators which may have been of significant benefit, had the data been available. For example, data in relation to literacy rates of adults can be a good indicator of the quality of primary education. However, this data was not consistently reported. Similarly, whilst the number of young people accessing secondary education was available, there was no data to suggest how many complete and to what level. The Department of Education should give thought to other indicators that may be useful to measure the countrys progress in relation to Education, which can then be used to more accurately direct future strategic planning. Once agreed, monitoring and reporting frameworks can be developed to capture and analyse the information gathered.

17

Part B

The second major part of the exercise involved the use of V.I.S.A. to carry out a Multi Attribute Value

Analysis (MAVA) to determine what course of action the Minister of Education should choose when

deciding on strategies to improve post-graduate education. The key steps involved in this process

are outlined below.

Value Tree

Post Graduate Education was chosen as the main criteria. We then set our first and second tier

branches as the following and shown in Figure 11 below:

Project Cost

o Initial Investment

o On-going expenses

National Benefit

Students

o Repatriation rate of Graduates

o No. of Candidates

o Graduation Rate

Institution

o Tuition & Living Cost per student

o Quality of Education

o Prestige of Institution

We developed our tier 2 options from a much larger pool of inputs. By role playing as the different

stakeholders we were able cut to the core of our concerns. These inputs were then reduced to the

final number by going through a mini MCDA exercise and determining which of these were actual

concerns within the decision making process, within this context. This allowed us to create our tier 1

groups. It was interesting to note that National Benefit was, on the surface, involved in so many of

the inputs that we decided that this had to be a tier 1 input on its own.

Figure 11 V.I.S.A. Value Tree

18

Scoring

Scoring for each of the items was based on some light research for figures. For those items ranked

using High/Medium/Low we used high as the analogy for best i.e. in the case of initial investment,

Option A was scored as high as this would have been the cheapest, or best of the alternatives. For

the 5 point scoring each alternative was scored in relation to each option. The summary of this

scoring can be seen in Figure 12.

Analysis

Structuring the Problem

We used the CAUSE framework to outline and sanity check that all angles of the problem had been

explored.

Criteria

From the briefing document it was established that the criteria for this problem was to allow the

Minister of Education to evaluate the feasibility of a number of options, or the need for further

options for increasing the number of graduates with analytical skills to compliment core subjects.

Figure 12 Scoring of Criteria

19

Alternatives

The alternative outcomes analysed were the 3 suggested by the Minister and a 4th best of all worlds

option:

Option A Send Students to Study Abroad

Option B Bring Teaching into Indonesia

Option C Develop Courses within Indonesia

Option D Combination

Option D was created to capture the best elements of the first 3 options. It was envisaged that a

programme would be created in which students were initially sent overseas to study whilst the

academic and physical infrastructure for delivering high quality degrees was developed. This

development would begin by bringing in high quality academic staff that would eventually be

replaced by native staff as the programme matured.

Uncertainties

There were many uncertainties discussed from the outset, these uncertainties ranged from either

lack of particular knowledge or data such as funding sources, through to variables such as type of

courses sought:

Funding it was discussed that in all likelihood this would be a combination of government

expenditure, NGO funding (such as the ADB) and local fund raising from existing institutions

wanting to be involved.

Participants it was unknown what the scale of participation would be. Would these

programmes only be offered to the academic elite? Would a lottery type system be used?

Would such a system be available to all? It was decided that this was an uncertainty which

would be clarified better by narrowing the options and so could be discounted until after the

feasibility analysis of the current options.

Industry involvement it was obvious that Industry would be included in the discussion but

what was uncertain was what their role would be. Some of the options discussed included

industry as sponsors of students, industry as the beneficiary of higher calibre graduates, or

industry as an investor in areas such as research hosted by universities. Again it was decided

that even with Industry involvement during the discussion on options, the actual extent of

involvement would be better determined once the options had been narrowed down.

Type of education although broadly defined by the Minister as more analytical, the types

of courses being pursued would have an impact on things like length of study or capital

investment. The level of degree (i.e. Doctorate, Masters or Bachelor) was also uncertain.

20

Stakeholders

In the briefing document, the Minister suggested 4 types of stakeholder:

Government

Industry

Local Colleges

Students

On discussion it was determined that at this stage it would not be prudent to include local colleges

as their vested interests would always bias them.

It was discussed if any major stakeholders were missing from this group. During this discussion it

was concluded that Government as a stakeholder was too broad a term to encompass the divergent

priorities of different areas. This led us to define government stake holders as Government

(Education), Government (Finance), and Government (Public Relations).

Government Education

Government Finance

Government Public Relations

Industry

Students

External / Environmental

Consideration to external influences led us into a broad discussion on culture and graduate

opportunities around the world. Although by no means exhaustive, the list of external factors that

we felt covered the majority of possible sources of interruption to our decision process were:

Availability of international student places in foreign universities.

Political relationships with foreign nations.

Natural disaster diverting funding.

Competition for graduates within host nations or other nations.

Change in government through democratic or subversive means.

Culture clash leading to non-completion.

Game Changer technology leading to decrease in the use of traditional teaching institution.

As mentioned, these factors are not exhaustive but it was felt that these could offer a sufficient

threat to the stability of the decision making process that they had to be acknowledged.

21

Weighting

When discussing the weighting for each of the inputs, the group took on the roles of the

Stakeholders. The discussion and decision for each of the inputs is outlined below. In order to

increase efficiency the weightings were split into tier 1 and tier 2, with tier 2 being discussed first

and each grouped according to their parent. Each input in a family was weighted against the other

inputs within that family and not within the context of the whole tree.

Tier 2

Project Cost

o Initial Investment 82.5%

o On-going expenses 17.5%

The discussion surround this element was based around the upfront costs of the project. It was

agreed that regardless of the option chosen there would be a certain amount of on-going cost.

However this could not be discounted completely as there would need to be a balance between how

the total cost would be distributed. A figure of $100m for 10 years was suggested as the basis for

discussion. This led to discussion on whether $10m every year would return a better investment

than say $80m in the first 2 years, then $2.5m each year for the remaining 8 years.

The stakeholders had to be reminded that this discussion was to be on the weighting within the

decision rather than on the detail of any implementation of any of the options.

Students

o Repatriation rate of Graduates 7%

o No. of Candidates 61%

o Graduation Rate 32%

When examining this component the various stakeholders seemed to quickly reach agreement on

the repatriation rate very quickly. It was discussed that there was little that could be done to

directly affect this trend as international statistics are pretty robust in their stability.

A bigger disagreement came about when discussing the number of candidates. Whilst some stake

holders i.e. the student, were of the opinion that the pool of potential candidates should be a large

consideration, the counter argument was that there would always be candidates available and that if

selection criteria were added this would naturally reduce the candidate pool. Agreement was

reached between the Industry and Government stakeholders that in considering return on

investment, volume was a primary consideration.

In regard to graduation rates, it was discussed that with the widening of the criteria in order to

increase the pool size, it would be likely that the calibre of the candidates would decrease thus

effecting graduation rates. It was decided that it was important to balance the graduation rates

against the number of candidates and so was weighted accordingly.

22

Institution

o Tuition & living Cost per student 45%

o Quality of Education 48%

o Prestige of Institution 7%

In discussing the Institution components, all stakeholders agreed that the prestige of the Institutions

the graduates were coming from was of minor importance, so long as the quality of the degrees

being achieved was sufficiently high, as this would normally correlate with a more prestiges

Institute. This then led to a discussion in which the quality of the education was decided as a high

weighting factor. The Industry stakeholder argued that they needed quality graduates with good

degrees, the government argued that there needed to be a quality component in order to sell the

whole project and the student agreed that quality degrees would have more attraction to individual

students.

The discussion then naturally turned to the cost of quality degrees. As it had been decided that a

quality degree would normally be associated with a prestigious institute then there would be a cost

premium. It was agreed by all that when looking at individual programs, the cost of them would play

a large part when scaling up by numbers and needed to be kept as a major consideration.

Tier 1

Once each of the components of the tier 1 inputs was weighted, discussion turned to the relative

importance of these and the associated weighting. Involved in this discussion was again the

Government with a financial advisor and a PR advisor, Industry representatives and a student body

representative.

Project Cost 46%

National Benefit 35%

Students 2%

Institution 17%

Discussion began by all parties agreeing that the overall cost of the project was the primary

consideration as without the ability to finance, the rest of the inputs were moot.

Secondly it was argued by both Industry and the Government that there needed to be national buy

in to the program and that there needed to be tangible benefits to the country. At this point

Industry stated that its primary consideration was in getting good graduates so they did not really

care where they were taught so long as they were of high quality. Government argued that there

was a need for internal investment in any case and so each option should be considered with this in

mind.

In discussing the student input there was general consensus that candidates could be found in any

scenario and so in the overall decision making process this was a negligible consideration when

weighted against the other inputs.

The last input, Institutions, was discussed at some length. Government saw this linking in with

National benefit when a home grown option was discussed and that sending Indonesian students to

23

well know institutes around the world would raise the profile of the quality of Indonesian graduates.

The student representative also agreed that there should be a non-trivial weighting attached to this

in the decision making process.

A summary of the weightings applied to each criterion given the above can be seen in Figure 13

below:

Outcomes

The outcome of the value tree indicated that of the listed alternatives, based on the criteria

selected, their scores, and the weighting associated with them, that the most desirable outcome, by

a very small margin, was actually option B Bringing in teachers to Indonesia.

This can be seen in the score summary charts in Figure 14.

Figure 13 Weighting of Criteria

24

By viewing the score profile across all of the options we can clearly see that when each input was

taken individually option B came out on top twice, and even then it was joint top with option A in

both criteria.

If we allocated a score to the position of 4 being top and 1 being bottom we see that option B is

again the best outcome but that options C and D are tied, as can be seen in Figure 15.

Figure 15 Alternatives sense check ranking table

Figure 14 Score Summary Charts from V.I.S.A.

25

Weight Sensitivity Analyses

When working through the various criteria there were two critical inputs which showed there would

be a dramatic change in outcome if the weighting were to be changed. This first of these, Initial

Investment, had from the beginning felt like a critical factor.

We can see from Figure 16 that with the exception of option D, the three remaining options all have

remarkably steep slopes with all options intersecting at various points.

At our agreed weighting we can see that the intersect of option B and option D is extremely close,

this is reflected in the overall ranking of the options where we see option D being a very close

second place.

When taking a step back from this and looking at the tier 1 input, Project Cost, we see this aggressive

divergence again emerge, and the same rankings reflected, as shown in Figure 17.

It is obvious from both common sense and from the evidence seen that project costs are a huge

contributory factor in any decision making, and the weighting placed upon it can completely reverse

the preferred outcome.

Figure 16 Weight sensitivity analysis of Initial Investment vs Post Graduate Education

Figure 17 Weight sensitivity analysis of Project Costs vs Post Graduate Education

26

The next criterion which was deemed as critical was National Benefit. It was felt that any issue

which could impact a nation from top to bottom and would have large cost implications would be a

very decisive issue.

On running the sensitivity weighting we can clearly see that this is another criterion which,

depending on the weighting placed on it by the stakeholders, could radically change the preferred

outcomes. Again by looking at the steepness of the slopes we can infer that it would only take a

small amount of change in the weighting of the criteria to change the outcome of the decision as can

be seen in Figure 18.

With the current weighting given to this criterion we can see that we are already very close to two

cross over points which completely reverse the ranking of the 4 options.

Recommendations

Option B and option D are so close that it is recommended that a second iteration of the process is

undertaken covering these as the alternatives, with a further set of criteria developed, in order to

examine the root drivers in more depth.

Whilst option B gives a definitive course of action, Option D would require a greater amount of prior

planning. This additional time in preparing an action plan for Option D may then become the main

differentiating factor. If desire for quick implementation is a primary consideration then Option B

then becomes the clear outright best fit option.

It was interesting to note that Option B was not an option any of the stakeholders were initially

biased toward and this could either strengthen the integrity or raise question marks over the scoring

and weighting process. This would also increase the argument for a second iteration for the decision

making process between Option B and Option D, as the scoring and weighting could be reviewed

across the stakeholders again to ensure there was absolute clarity over what was important to each

stakeholder.

Figure 18 Weight sensitivity analysis of National Benefit vs Post Graduate Education

27

Another conclusion which is important to draw from this exercise is that biased opinion inherently

driven by human nature, cannot lead to a purely objective result. Although the importance of

including those biased players (as they are the stakeholders) is not in doubt, it is where their input is

applied that must be constantly considered and questioned.

This emphasises the importance of a facilitator figure to take charge of the process in order to

redress any bias which is inserted into the value tree build. This facilitator must objectify as much as

possible the reasoning behind any scoring not directly supported by independent fact and also in

allocating levels of relative importance.

In any case, the exercise acts as a valuable tool to aid the Minister in eliminating options which are

not feasible and improves the level of focus on those options that indicate a positive step forwards.

28

References

AngloINFO. (2013). Primary and Secondary Education in Indonesia. Retrieved February 14, 2013,

from AngloInfo: the global expat network: http://indonesia.angloinfo.com/family/schooling-

education/primary-secondary/

Hashtag, T. (2013). ASDM Assignment Excel Workbook. Glasgow.

The World Bank Group. (2013). World Bank Data. Retrieved January 20, 2013, from The World Bank:

http://data.worldbank.org/topic/education

29

Appendix 1 Task List

Group member

Task Owen Paterson Martin Provan Karen Taylor Wes beard

Chellappan

Thirunavukkarasu

PART A

Data Cleansing worked together as group worked together as group worked together as group

worked together as

group

worked together as

group

Identifying Gap in Available

Data worked together as group worked together as group worked together as group

worked together as

group

worked together as

group

Additional Data Collection worked together as group worked together as group worked together as group

worked together as

group

worked together as

group

Data Preparation &

Calculation worked together as group worked together as group worked together as group

worked together as

group

worked together as

group

Data Visualization worked together as group worked together as group worked together as group

worked together as

group

worked together as

group

Exploratory Analysis discussion of results discussion of results discussion of results Lead discussion of results

Inference Analysis discussion of results discussion of results discussion of results discussion of results Lead

Regression analysis discussion of results discussion of results discussion of results discussion of results Lead

Recommendations worked together as group worked together as group worked together as group

worked together as

group

worked together as

group

30

PART B

Problem structuring Facilitator role play role play role play role play

Scoring and weighting using software using software worked together as group

worked together as

group

worked together as

group

Analysis Lead worked together as group worked together as group

worked together as

group

worked together as

group

Recommendation Lead worked together as group worked together as group

worked together as

group

worked together as

group

Report Lead Proof read and comments Proof read and comments

Proof read and

comments

Proof read and

comments