international advisory review of the swedish ... - vr.se · were erik fledderus (director at surf,...

VETENSKAPSRÅDET 2017

INTERNATIONAL ADVISORY REVIEW OF THE SWEDISH NATIONAL

INFRASTRUCTURE FOR COMPUTING (SNIC)

INTERNATIONAL ADVISORY REVIEW OF THE SWEDISH NATIONAL INFRASTRUCTURE FOR COMPUTING (SNIC)

VETENSKAPSRÅDET

Box 1035

SE-101 38 Stockholm

VR1703

ISBN 978-91-7307-346-2

International advisory review of the Swedish National Infrastructure for Computing (SNIC)

FOREWORD The Swedish Research Council (SRC) is a public agency under the Ministry of Education and Research. The council has been mandated to prioritise and fund national research infrastructures (RIs) and participation in international organisations and RIs. The overall objective of the SRC is to fund excellent science and therefore all prioritisations, including participation, investments and access to different RIs, are based on scientific excellence.

The council of research infrastructures (RFI) is one of the decision making boards within the SRC1. RFI is responsible for policy-making, strategy development and funding of RIs of national interest. The funding is competitive and allocated through bi-annual calls that are based on a national roadmap. In 2015, the SRC initiated the implementation of a new model for prioritising and funding national RIs, where the universities are given more responsibility, not only for co-funding but also for prioritising and governing national RIs. The model has been developed by the SRC together with the ten major research universities, through a high-level reference group (URFI). One important purpose of this new scheme is to engage and involve the universities in supporting the RIs in order to achieve long-term sustainability.

The use of the Swedish National Infrastructure for Computing (SNIC) has increased dramatically over the past years. This has put SNIC under pressure to deliver services and user support to various scientific communities, as well as other RIs, with varying requirements and demands on the available resources. In order to develop SNIC further and enable it to meet the new challenges, RFI has asked SNIC and its host university – Uppsala University – to provide a strategic plan and budget as a basis for the council’s next funding decision. This has been done in close collaboration with the URFI group. To support them in this process, and for us to receive advice on further steps to be taken, an advisory panel was recruited. The main objective of the advisory panel was to provide an independent analysis of the preliminary strategic plan and budget. This report is the result of the panel’s work, and their comments and recommendations will be a valued and important contribution to the continued work of SRC, Uppsala University, URFI and SNIC.

The advisory panel was chaired by Charlotte Jamieson (Senior manager at STFC, UK). Other panel members were Erik Fledderus (Director at SURF, Netherlands), Ruben Kok (Director at DTL, the Dutch Tech Centre for Life Science, Netherlands) and Kenneth Ruud (Pro-rector of research at Tromsø – The Arctic University of Norway, Norway). I would like to extend my gratitude to the panellists for supporting this process and for their great contribution to this work. Their collective expertise and insight are truly valuable and important to us! Stockholm, 2017-02-16 Björn Halleröd Secretary General of the Council for Research Infrastructures

1 Throughout the report the panel does not distinguish between the SRC and the RFI in the text and recommendations.

CONTENT

FOREWORD ................................................................................................................................................. 2

EXECUTIVE SUMMARY ............................................................................................................................... 4

SAMMANFATTNING ..................................................................................................................................... 5

BACKGROUND AND OBJECTIVES ............................................................................................................. 6

INTERNATIONAL ADVISORY PANEL ......................................................................................................... 7Members of the advisory panel ................................................................................................................. 7Deliverables from the panel ....................................................................................................................... 7Assessment questions .............................................................................................................................. 7

THE PANEL’S WORK PROCESS................................................................................................................. 8Information provided to the panel .............................................................................................................. 8The next steps of the review process ........................................................................................................ 8

OBSERVATIONS AND RECOMMENDATIONS FROM THE ADVISORY PANEL ....................................... 9The transition from SNIC to SNIC 2.0 ....................................................................................................... 9Broadening the scope of SNIC 2.0 ............................................................................................................ 9Broadening the ownership of SNIC 2.0 ................................................................................................... 11Governance ............................................................................................................................................. 12Budget ..................................................................................................................................................... 12SWOT analysis ........................................................................................................................................ 13Further measures to be strongly considered by the SRC ....................................................................... 14Final words .............................................................................................................................................. 15

INTERNATIONAL ADVISORY REVIEW OF SNIC 4

EXECUTIVE SUMMARY The panel commended the advisory process and was impressed by the close cooperation between the universities (through URFI2) and between the URFI and SNIC. In the transition from the old SNIC to the new SNIC 2.0, and for SNIC 2.0 to become successful, the panel stressed the following key aspects to be addressed by Uppsala University together with URFI:

• A clear vision has to be developed and disseminated by all stakeholders to gain acceptance for the transition

at hand. Important topics for such an enhanced vision are listed below. • SNIC 2.0 should take on a national leadership role for developing an adequate and comprehensive e-

infrastructure to support excellent science. This requires building relationships with universities as well as other e-infrastructures and data-producing RIs.

• A broadened scope for SNIC 2.0 is necessary for the infrastructure to continue to be relevant for its users; this relates particularly to addressing different aspects of the data life cycle.

• A broadened ownership for SNIC 2.0 is necessary. Universities, researchers, science communities, other RIs, SNIC-staff etc. should have a sense of ownership of the organisation. This is important to establish, since the consortium and user communities for SNIC 2.0 will extend beyond the universities which currently have HPC centres.

• An improved implementation plan for the new organisation SNIC 2.0 has to be developed. It should demonstrate a clear vision for the organisation as well as the steps needed to move from SNIC to SNIC 2.0.

• A budget needs to be developed, including not only monetary contributions but also transparent pricing of in-kind contributions. The budget should be accompanied by a set of metrics that can be monitored to ensure that the operation of SNIC 2.0 is cost-effective and of high quality.

• A risk analysis needs to be developed to demonstrate an awareness of the risks in moving from SNIC to SNIC 2.0, as well as implementing and operating SNIC 2.0. The risk analysis should be accompanied by proposed measures to mitigate these risks as much as possible.

• The governance structure for SNIC 2.0 has to be further developed to ensure a sustainable and stable organisation that is able to take strategic decisions, make prioritisations and provide sufficient commitment in case of difficult economic conditions.

All of the above are important to consider and the last four are absolutely crucial to provide, as without these the SRC may not be able to take a funding decision. The panel also recommended the SRC to: • Monitor and evaluate the progress of implementing SNIC 2.0 based on the above/mentioned documents and

metrics. This could be done through regular (for example twice a year) meetings between the SRC and the host university.

• Lead the development of a national plan for a coherent e-infrastructure in Sweden, including all aspects of network, HPC and data. The plan should include the further development of SNIC 2.0, and also the network infrastructure SUNET and domain-specific e-infrastructures (such as ELIXIR and WLCG). The national plan needs to be developed in close collaboration with URFI and other stakeholders.

• Consider how software development and sustainability can be supported on a national level as part of the broader e-infrastructure of the future. SNIC resources can be used more cost-efficiently if scientists and RIs can write and use optimised code and middleware.

2 The URFI group is composed of the ten major research universities, which are Chalmers Technical University, Gothenburg University, Karolinska Institute,

Linköping University, Lund University, Royal Institute of Technology, Stockholm University, Swedish University of Agricultural Sciences, Umeå University and

Uppsala University.


SAMMANFATTNING Panelen berömde rådgivningsprocessen och var imponerad av det nära samarbetet mellan universiteten (via URFI3) och mellan URFI och SNIC. I övergången från det gamla SNIC till det nya SNIC 2.0, och för att SNIC 2.0 ska bli framgångsrikt, betonade panelen de följande nyckelaspekterna som måste beaktas av Uppsala universitet tillsammans med URFI. • En klar och tydlig vision måste skapas och spridas av alla intressenter för att den kommande övergången ska

bli accepterad. Viktiga ämnesområden för en sådan förstärkt vision anges nedan. • SNIC 2.0 bör anta en nationell ledarroll inom utvecklingen av en adekvat och heltäckande e-infrastruktur

som stödjer förstklassig vetenskap. Detta kräver att man skapar relationer med universitet och även med andra e-infrastrukturer och dataproducerande forskningsinfrastrukturer.

• SNIC 2.0 kräver en större räckvidd för att infrastrukturen ska förbli relevant för användarna – detta handlar i synnerhet om att beakta olika aspekter av datalivsomloppet.

• Ett bredare ägande av SNIC 2.0 krävs. Universitet, forskare, forskarsamhällen, andra forskningsinfrastrukturer, SNIC-personal osv bör ha en känsla av att de äger organisationen. Detta är en viktig känsla att skapa, eftersom SNIC 2.0:s konsortium och ägargrupper kommer att sträcka sig utanför de universitet som för närvarande har HPC-centrum.

• En förbättrad genomförandeplan för den nya organisationen SNIC 2.0 måste utvecklas. Den bör demonstrera en klar vision för organisationen och även de steg som krävs för övergången från SNIC till SNIC 2.0.

• En budget måste fastställas som inkluderar inte bara penningmässiga bidrag utan också transparent prissättning och ”in kind”-bidrag. Budgeten bör åtföljas av ett antal mätbara indikatorer som kan följas för att säkra att SNIC 2.0 drivs på ett kostnadseffektivt och högkvalitativt sätt.

• En riskanalys behöver tas fram för att demonstrera att man är medveten om riskerna med övergången från SNIC till SNIC 2.0, och även med att genomföra och driva SNIC 2.0. Riskanalysen bör åtföljas av föreslagna åtgärder för att minska dessa risker så mycket som möjligt.

• Styrningsstrukturen för SNIC 2.0 måste utvecklas vidare för att säkra en hållbar och stabil organisation som kan fatta strategiska beslut, göra prioriteringar och tillhandahålla tillräckligt engagemang i det fall svåra ekonomiska förhållanden uppstår.

Alla de ovanstående är viktiga att beakta, och att tillhandahålla de sista fyra är helt avgörande, eftersom Vetenskapsrådet (RFI) inte kan fatta något finansieringsbeslut utan dessa. Panelen rekommenderar också att Vetenskapsrådet: • Följer och utvärderar hur genomförandet av SNIC 2.0 framskrider baserat på de ovanstående dokumenten

och måtten. Detta bör göras genom regelbundna (t ex två gånger per år) möten mellan Vetenskapsrådet och värduniversitetet.

• Leder utvecklingen av en nationell plan för en sammanhängande e-infrastruktur i Sverige, inklusive alla aspekter av nätverk, HPC och data. Planen bör innefatta den fortsatta utvecklingen av SNIC 2.0 samt också nätverksinfrastrukturen SUNET och domänspecifika e-infrastrukturer (t ex ELIXIR och WLCG). Den nationella planen bör utvecklas i nära samarbete med URFI och andra intressenter.

Tänker på hur programvaruutveckling och hållbarhet kan stödjas på nationell nivå som en del av en bredare framtida e-infrastruktur. SNIC:s resurser kan användas mest kostnadseffektivt om forskare och forskningsinfrastrukturer kan skriva och använda optimerad kod och mellanprogramvara.

3 URFI-gruppen består av de tio främsta forskningsuniversiteten, som är Chalmers Tekniska Universitet, Göteborgs Universitet, Karolinska Institutet, Linköpings

Universitet, Lunds Universitet, Kungliga Tekniska Högskolan, Stockholms Universitet, Sveriges Lantbruksuniversitet SLU, Umeå Universitet och Uppsala

Universitet.


BACKGROUND AND OBJECTIVES The second largest (in terms of funding) national RI is the Swedish National Infrastructure for Computing (SNIC). SNIC provides computing, storage and advanced user support to Swedish researchers and is hosted by Uppsala University. SNIC also participates in a number of international organisations and collaborations, such as EGI, EUDAT, NeIC, PRACE and WLCG.

The demand for services provided by SNIC has increased dramatically in the last couple of years as a result of many scientific disciplines becoming increasingly data-intensive, and this trend is expected to continue. Furthermore, the construction and upgrading of data-producing national RIs, such as Max IV and SciLifeLab, have further increased the demand for an adequate e-infrastructure that supports both the operation of other RIs and analysis of the data generated at these RIs.

The need for e-infrastructure such as SNIC is evident, and the SRC expects to continue to fund this type of services. To this end, the SRC has already decided that SNIC should not apply in the bi-annual infrastructure call but instead be handled in a separate process leading up to a funding decision in autumn 2017. Furthermore, the SRC has stated that it expects to fund SNIC at approximately the current level, which is 100 million SEK/per year. The synchrotron source Max IV, which is the largest national RI, is handled in a similar way. The justification for this separate process is that the SRC has already stated that it will fund part of the operation for both of these RIs, so the evaluation process leading up to a funding decision should rather focus on establishing an appropriate level of engagement for the SRC.

The funding decision for SNIC will be based on a strategic plan and budget containing the following components: a) the scope and range of activities within SNIC; b) the user base of SNIC; c) the process for prioritisation of investments and users; d) the level and mechanisms for co-funding; e) the consortium supporting SNIC; f) the organisation and leadership of SNIC; and g) the technical implementation. The strategic plan and budget should cover a five-year period from 2018 to 2022, and the final version of these documents will be considered by the SRC to be equivalent to the content of an ordinary proposal in the research infrastructure call. The only difference is that the strategic plan and budget are shaped in an iterative process, where the documents are submitted to the SRC twice and the SRC will provide feedback after the first submission. The feedback will be provided through an advisory panel with international experts.

The purpose of the iterative process is to develop a strategic plan and budget of as high a quality as possible. Since the SRC has already stated that SNIC is an essential RI which will receive continued funding, the process leading up to a funding decision will focus on the transformation of SNIC into an e-infrastructure capable of matching the increasing demands from different user groups. The purpose of the evaluation by the advisory panel was therefore twofold. The advisory panel provided input and recommendations for Uppsala University (and URFI) on how to improve the proposed establishment and operation of SNIC 2.0. Furthermore, the panel provided recommendations to the SRC on how to monitor and evaluate the process of establishing SNIC 2.0 and follow-up of the e-infrastructure. The panel also added some more general recommendations to the SRC as a whole.


INTERNATIONAL ADVISORY PANEL

Members of the advisory panel The panel members were recruited to provide a collective competence range covering strategic, organisational and technical aspects of operating, managing and funding e-infrastructures. The advisory panel was led by Charlotte Jamieson (senior manager at STFC). The other members were Erik Fledderus (Director at SURF), Ruben Kok (Director at DTL, the Dutch Tech centre for Life science), and Kenneth Ruud (Pro-rector of research at Tromsø – The Arctic University of Norway).

Deliverables from the panel The panel was asked to deliver a short report of approximately ten pages, where the following bullet points were addressed. The report summarises the discussions and conclusions from the panel’s assessment and includes recommendations for the host university, relevant stakeholders and the SRC.

Assessment questions The panel was asked to assess and comment on the following: • Is the preliminary strategic plan reasonable in terms of: strategy, priorities, organisation, implementation and

technical aspects? How does it compare to other similar infrastructures? • Is the preliminary budget reasonable? How does it compare to other similar infrastructures? • Is it possible to implement the proposed strategic plan within the proposed budget? • If the proposed strategic plan and budget are to be implemented: o What are the strengths, weaknesses, opportunities and threats? Please comment on strategic,

organisational, financial and technical aspects. o Which measures could be taken, by Uppsala University and the SRC, to mitigate the observed

threats/risks? o Which measures could be taken by Uppsala University to improve the proposed strategic plan and budget? o Which measures could be taken, by Uppsala University and the SRC, to ensure successful implementation

of the proposed strategic plan and budget? o Which aspects need to be monitored by the SRC:

• when reviewing the final strategic plan and budget in spring? • during the later implementation of the proposed SNIC 2.0?


THE PANEL’S WORK PROCESS

Information provided to the panel The advisory panel was provided with the following material before the panel meeting.

- Terms of Reference (ToR) for the work. - A briefing paper describing the model for funding of research infrastructures of national interest, an

introduction to SNIC and other national e-infrastructures, and finally the SRC’s intentions for future funding of e-infrastructure including SNIC.

- The report from the previous evaluation of SNIC in 2010. - A summary of SNIC’s user survey 2016, which was produced by the SNIC head office. To ensure transparency in the process, the advisory panel and Uppsala University received the same information and documents.

In addition to the written material, the panel also had the opportunity to meet and discuss with a number of stakeholders to gain further insight and ask questions. Björn Halleröd (Secretary General of the RFI) and Ingela Nyström (Vice-Chair of the RFI) provided further information on prioritisation and funding of RIs in Sweden, as well as insight into the mandate, goals and vision of RFI. The Director of SNIC, Hans Karlsson, described the current state of SNIC. Half a day was set aside to a hearing, where representatives from Uppsala University, URFI and SNIC participated. The invitation to the hearing was send to Uppsala University, as the host of SNIC, and they selected the participants for the hearing. The following persons participated: - Kristina Edström, the Vice-Chancellor’s special advisor on research infrastructures at Uppsala University

(the host university for SNIC), member of URFI. - Per Dannetun, Director of Research at Linköping University, member of URFI. - Anders Karlhede, Dean of Natural Sciences at Stockholm University, member of URFI. - Hans Karlsson, Director of SNIC. - Sverker Holmgren, Chair of the group responsible for writing the strategic plan and budget on behalf of

URFI. - Stacey Sörensen, Deputy Vice-Chancellor of Lund University, Chair of URFI. The panel regretted that the Chair of the Board of SNIC was not able to attend the hearing. The presentations and discussions during the hearing provided a substantial amount of additional information to the advisory panel, as well as guidance and input to the participants from SNIC’s host university, URFI and SNIC. The mutual exchange of information and experience was perceived as very positive by all participants in the hearing.

The next steps of the review process Uppsala University will receive the panel report and will be asked to consider the comments and recommendations in preparing the final version of the strategic plan and budget for the SRC (to be submitted in April 2017). By then, it is expected that the strategic plan and budget is fully supported by URFI, whose members are the same as those of the proposed consortium for SNIC 2.0. To anchor the report at the SRC, the chair of the advisory panel will be invited to present the conclusions and recommendations of the panel at the RFI meeting in March 2017. The final submission from Uppsala University will be reviewed in an in-house process at the SRC, where this report from advisory panel will be used as the foundation. The RFI anticipates that only minor comments will be made at this stage.


OBSERVATIONS AND RECOMMENDATIONS FROM THE ADVISORY PANEL The panel has addressed and thoroughly discussed all assessment questions (see subsection Assessment questions). However, since some of the questions overlap, the observations and recommendations below are instead structured around a number of thematic headings.

The transition from SNIC to SNIC 2.0 The panel welcomed the review process initiated by SRC, and was looking forward to a positive outcome of the constructive dialogue between the funding agency and the stakeholders. The panel recognised that SRC has created an open and accepted process, which has developed trust among the partners. The strategy and vision presented in writing and at the hearing was overall well-received by the advisory panel. The panel specifically liked the close relationship between URFI and SNIC, which is commendable, given the short time frame for formulating a strategic plan. The URFI platform should continue to be used for a dialogue between the universities, and also for the current SNIC board, to empower a smooth transition into SNIC 2.0.

The panel recognised the challenges in transitioning the current SNIC organisation into the new infrastructure, described as SNIC 2.0. It was emphasised by the panel that it is not enough to only describe the new SNIC 2.0, but that it is equally important to demonstrate in the strategic plan and budget how the step-wise implementation will be realised. In order to enable the stakeholders to follow the progress, there is a need to develop milestones and metrics that are accepted by all partners as indicators of a successful implementation of SNIC 2.0. It is important that such milestones and metrics cover all aspects of the infrastructure, and all types of services offered. The mature SNIC 2.0 infrastructure will need to be able to demonstrate that it delivers cost-effective services of high quality that support excellent science in order to protect and potentially grow its budget to meet future increasing demands. Part of this will be to demonstrate not only technical excellence and efficiency, but also demonstrate commitment to values, such as training, the development of users’ skills and the ability to create a sense of ownership and community around SNIC 2.0. These non-technical aspects will all provide added value. Regular meetings (for example twice a year) between the SRC and the host university are strongly encouraged as an instrument for the SRC to support, monitor and evaluate the progress of implementation, and later the sustained operation, of SNIC 2.0. In the early stages, the governance structure will be an important aspect to follow for the SRC, since a stable and accepted governance structure is a prerequisite both for a smooth transition to SNIC 2.0 and for maintaining a long-term sustainable relationship with the stakeholders.

The panel was happy to see SNIC’s awareness and alignment with other initiatives that are ongoing, nationally, in the Nordic countries and internationally. The dialogue within the URFI group, as well as between URFI, the SNIC board and the SRC, should work to continue the alignment of SNIC 2.0 with other national, Nordic and international initiatives. This is important in order to make cost-effective use of the funding and also as a way of creating synergies that are valuable to SNIC 2.0. Taking on leading roles in international RIs and organisations is a way of building a strong reputation for SNIC 2.0, although the investments in time and money always need to be balanced against the costs.

The panel commended and encouraged the good work already done by Uppsala University and URFI. As host to the infrastructure, Uppsala University has demonstrated a strong lead in driving the process this far, and has to continue to do so. In order to successfully implement SNIC 2.0, the panel emphasised that Uppsala University and URFI have to set up a strong board with a national mandate. Such a board should further develop and sustain the new vision towards SNIC 2.0, and support the SNIC Director in implementing this vision.

Broadening the scope of SNIC 2.0 The transition from the old SNIC organisation to a new SNIC 2.0 includes a broadening of the scope of the RI. The old SNIC has focused on HPC, but as many fields of science and innovation are rapidly developing into data-intensive fields, it is important to recognise that HPC is not enough, and the panel emphasised that the new


SNIC 2.0 should encompass a broader portfolio. The panel congratulated Uppsala University and URFI on recognising this need for change, and encouraged them to continue on this path and to take responsibility for realising the necessary transition. Most likely, this change could not grow incrementally from the current SNIC organisation, but will require some fundamental changes, as reflected in the fact that the proposed SNIC 2.0 will require a very different organisation. To be successful in this transition, there is a need for a shared and clear vision which is accepted and disseminated by all stakeholders. As a way of communicating clearly that SNIC 2.0 is a new e-infrastructure, a change of name could be considered.

The panel noted that many aspects of data management were defined as being outside the remit of SNIC 2.0. This was questioned by the panel, which was of the opinion that computing is just one part of the full data lifecycle, and should not be singled out if it is to evolve in line with other parts of the data lifecycle. The panel suggested that an expanded remit for SNIC should thus include different aspects of data, as well as expanded activities in training, software development and alignment with other RIs. Only then will SNIC 2.0 be a future-proof national e-infrastructure of high relevance for scientists, science communities and RIs. This implies that SNIC 2.0 will have to bring in additional competence. The panel noted a certain reluctance for SNIC 2.0 to take on a leading role in all of these areas. The reasons for this were unclear, and the panel strongly encouraged all stakeholders to support and empower SNIC 2.0 to become a national leader, which can drive development and become the preferred partner for scientists, science communities and RIs alike. Since the panel only focused on SNIC, it recognised that there may be unknown aspects that limit SNIC, in which case the SRC and URFI need to agree on how to proceed. There is evidently a need for a coordination entity that can take a lead in developing the ecosystem of services needed to support the full data lifecycle. The SRC, together with the URFI, is advised to consider who this should be, if it is not SNIC 2.0. If Sweden does not address the data challenges, this will severely hamper research nationally. As a consequence, Sweden will not be able to realise the full potential of the investments already made in research projects and RIs.

SNIC has a strong background in traditional HPC, and in serving science communities with a relatively high level of competence in computing and software development. For SNIC 2.0 to be successful, it has to embrace a slightly new role that also welcomes new users and user communities. To this end, the concept of distributed user support was well-received by the panel, as a way of providing support close to the users, providing local capacity-building, and also as a key element for including the four new members in the consortium. The presence of support staff at the ten universities in the proposed consortium is essential for building a relationship with the scientists and providing relevant know-how and expertise. With regards to this, the panel discussed the responsibility of the universities to employ skilled persons and to provide alternative career paths in order to encourage them to remain in academia in other roles than as scientists. The universities’ capability to make strategic plans for this is crucial if SNIC 2.0 is to be able to provide high-quality user support across the country. This will require close collaboration with the universities and, if implemented well, holds great promise to provide synergies between universities in terms of training and education of both current SNIC users and new and future generations of scientists. Even though the support staff will be employed at different universities, it is important that the head office for SNIC 2.0 has a centralised plan for the services, in order to uphold common standards and encourage synergies between universities. This can be ensured through service-level agreements for specific services.

SNIC 2.0 also needs to develop mechanisms for encouraging professionalised usage of resources. This requires both training of users and incentives for developing efficient and optimised code. User support could include advice on analysis strategies, as well as code development, optimisation and sharing, and also other activities. Test environments for trying out new methodology and techniques could also be considered. The panel noted that there are ongoing activities, for example at Nordic level, and recommends that SNIC 2.0 aligns its activities with these.

In parallel with the expansion of scope and remit of SNIC 2.0, the process for allocating resources to the users has to become more transparent. It should be transparent and clear to users how allocations are made. It became evident to the panel during the hearing that there can be a problem in coordinating allocation of time for researchers performing experiments at Max IV and SciLifeLab, and then subsequently being allocated SNIC resources for data analysis. This needs to be addressed to make sure that analysis of data does not become a severe bottleneck for research, while still making sure that scientific excellence and efficient use of the computational infrastructure is not compromised. To this end, the panel emphasised that even if allocations of experiment time and SNIC resources are coupled, it is important to create some type of two-step evaluation, particularly in terms of assessing that the users have the right training, skills and code to use SNIC resources efficiently. The SRC should explore mechanisms for coupling allocation of time on data-producing RI and the


e-infrastructure needed to analyse the data. Working out the details of how this can be done will probably require collaboration between the SRC and data producing RIs, SNIC and URFI. Although the problem seems to be acute for Max IV and SciLifeLab, this is a generic issue that is likely to arise also for other RIs. It is therefore important to address it in a way that fits into the SRC’s, and other funding agencies’, funding schemes.

Furthermore, the panel encouraged SNIC to find mechanisms to stimulate and train new users and user communities to apply for access to its resources. For example, in the Netherlands, SURF sets aside a designated part of its budget to broaden the scope and allow for new user communities, innovative and novel technologies and services. This type of investments, to build for the future, would be strategic and should be part of the planning process for SNIC 2.0. It could also be an important way to achieve user involvement, to pick up new services, and to identify services that need to be phased out.

Open data and data stewardship are becoming increasingly important for science, and the panel strongly recommended the stakeholders to consider how this could be addressed nationally, both with regards to SNIC 2.0 and on a broader level. This global transition should result in a national policy and find implementation in ongoing science and infrastructure initiatives, both nationally and internationally. The European Open Science Cloud (EOSC) is one such initiative, which has to be on the national radar. The panel encouraged SNIC 2.0 to be proactive and take a leading role in data by including data stewardship in its strategy. This could preferably be accompanied by an effort of by the SRC to include data management plans in proposals for research projects (and RIs, if not already in place).

Broadening the ownership of SNIC 2.0 The proposed consortium for SNIC 2.0 has ten members4, an extension of the previous six universities that were previously part of the organisation. Broadening the consortium was seen as very positive by the panel, as a way of emphasising the willingness to bring in new user communities, as well as bringing in more funds. The panel learned that the universities within URFI together cover > 90% of the research in Sweden, so the proposed consortium has a wide uptake. The close relationship between the current national HPC centres was noted by the panel as positive, and this relationship is important to maintain also in SNIC 2.0, where it is expected that hardware will be physically located only at a limited number of sites. The number of sites with hardware constitutes a balance between efficiency and sense of ownership of the common infrastructure. Since computer hardware will not be physically located at all consortium members, it is crucial to build a sense of ownership based on other aspects than technology and hardware, such as a common vision, training efforts, local user support, etc. Broadening the portfolio of the envisaged SNIC 2.0 could potentially lead to alternative specialised roles for the individual institutes in the consortium.

Support staff at all universities in the consortium were seen as an important part in establishing common ownership by all universities. This set-up would also enable support staff to feel a sense of ownership and identification, both to a local SNIC-node and to the national infrastructure as a whole. To manage this distributed infrastructure, the panel advised that the number of staff working for the head office needs to increase. However, it may be envisioned that some staff working with common tasks are also employed at other universities than Uppsala, as long as there is a clear mandate for the SNIC Director to direct their work. Increased coordination of activities and strategic decisions by the head office is needed to avoid duplication of efforts and ensure cost-effectiveness in the infrastructure as a whole. It is crucial that the head office has a sufficient mandate by the board to do this.

Developing a close relationship with other data-producing and data-handling RIs is crucial, in order to capture their needs and integrate them in the planning cycles of both SNIC 2.0 and the other RIs. It is thus essential that SNIC 2.0 works in close alignment with RIs that are on the national roadmap and/or other RIs that have invested in SNIC 2.0. The panel noted that closer collaboration with SNIC 2.0 is important for all RIs, and in particular for the SciLifeLab and Max IV, which both seem to depend heavily on SNIC already. Uppsala

4 Chalmers Technical University, Gothenburg University, Karolinska Institute, Linköping University, Lund University, Royal Institute of Technology, Stockholm

University, Swedish University of Agricultural Sciences, Umeå University and Uppsala University.


University, as the SNIC host, is also part of SciLifeLab and could take responsibility for aligning this RI further, and support collaboration by formulating and disseminating a clear and common vision. SNIC 2.0 needs to collaborate closely with domain-specific e-infrastructures (such as ELIXIR and WLCG) to ensure that their efforts and activities are aligned. It will be important that SNIC 2.0 is inclusive to avoid some stakeholders building their own separate systems. In this process, it is important that URFI is committed to SNIC 2.0 as the instrument for providing e-infrastructure services on a national level. The reasoning behind this is that joint investments into and operation of e-infrastructure should be the most cost-effective solution for Sweden as a whole, and, with the information at hand, SNIC 2.0 seems to be the best instrument to achieve this.

Being inclusive of user groups and RIs with funding from other sources than the SRC is important for the success of SNIC 2.0 as a truly national e-infrastructure. To this end, the panel noted that in order to develop SNIC 2.0 into an effective e-infrastructure, everyone has to contribute to and to build the proposed SNIC 2.0 environment. Creating this sense of ownership among all stakeholders is the key to success. All RIs and user groups that use SNIC 2.0 have to contribute to the development of the infrastructure.

Governance The panel identified that one key to success is to establish a transparent and accepted governance structure for SNIC 2.0, including not only the board but also other advisory bodies. As presented now in the strategy document, the governance structure is incomplete and needs refinement in terms of how the stakeholders, board and advisory groups or committees are staffed and operate together (terms of reference for each). The governance structure must be robust enough to allow SNIC 2.0 to make strategic decisions, develop priorities and provide sufficient commitment also in the event of harsh economic conditions. The board needs to include sufficient competence in managing and operating a large RI, in terms of financial and legal aspects as well as technical aspects.

It is very positive that the governance concept presented involves broad representation of different universities. However, Uppsala University – as host – needs to set up and empower a governance structure with a clear role and responsibility across hierarchies, in order to embed and ensure transparency and buy-in from other stakeholders. To become accepted by all stakeholders, the governance structure should have selection and reporting processes that are transparent and fair, as well as clearly defined roles for monitoring and financial decisions.

Budget The preliminary budget provided to the panel was considered as probably reasonable, but the stakeholders will have to do some prioritisation. In order to do this, there is a need to develop metrics to measure cost-efficiency, resource-efficiency and also risks associated with the plan. SNIC 2.0 needs to have a detailed investment plan that covers system renewal cycles. The investment plan should also set up ways for monitoring and evaluating delivery of services and use of performance indicators as an aid in controlling costs and maximising use of the systems.

The budget has to be transparent to ensure trust between the consortium members and other stakeholders, and this is particularly true for in-kind contributions. To reduce costs, the panel suggested that SNIC 2.0 and URFI explore the possibility of centralised purchasing. They should also closely follow technology and market trends to enable them to be pro-active in their planning.

The preliminary budget submitted appears to be slightly lower than other similar infrastructures in similar countries. It was argued at the hearing that SNIC 2.0 requires additional funding from the SRC, but the panel did not see that this was convincingly demonstrated in the documentation provided. The broadening of activities and services in SNIC 2.0 is likely to require additional funding, but, on the other hand, this also offers opportunities to bring additional funding from new consortium members, users and RIs. In order to be the preferred partner, SNIC needs to demonstrate that it provides cost-effective services. One way of doing this is through benchmarking against similar organisations, using standard metrics and methods.

The proposed strategic plan can probably be implemented with the proposed budget. However, the material presented contained only enough details on HPC activities, and nothing on other types of activities (such as data). This information needs to be provided for the next submission and be aligned with the prioritisations for


SNIC 2.0. The stakeholders in SNIC 2.0 and the SRC are encouraged by the panel to explore the introduction of user fees to contribute to a sustainable model of operation and innovation of the infrastructure.

Uppsala University and URFI should consider creating a rolling five-year vision, which could be linked to the five-year periodic funding scheme. This would provide a useful tool for long-term planning.

SWOT analysis The panel was asked to comment on the strengths, weaknesses, opportunities and threats that it saw in the strategic plan, in terms of strategy, organisation, finances and technology. In the next version of the strategic plan, weaknesses and threats need to be addressed/mitigated, particularly the issues related to the budget. The following bullets summarises the SWOT analysis.

Strengths • There is a strong relationship within URFI that is essential for the transition from SNIC to SNIC 2.0, as it

builds co-ownership and provides critical mass (90% of the research and RI) and has a broad view of what is needed for delivering excellent science.

• The support from URFI can provide long-term sustainability in terms of funding. • SNIC is a trusted partner with good collaboration between the head office and the current SNIC centres. This

is a good foundation to build on. • The organisation has formulated a good first version of the vision for SNIC 2.0, which aligns well with

international initiatives. • SNIC has high technical skills. • SNIC and its partners have good and relevant international contacts.

Weaknesses (which if not addressed have the potential to become threats) • There was a lack of detail in the organisational and governance structure, which is understandable given the

short time frame for implementation, but this must be developed for the next stage of the process. • SNIC seems to perceive artificial boundaries in terms of mandate and scope, which is holding it back and

restricting initiative, ownership and leadership. • There was no evidence of risk analysis in the strategic and budget plan. • The strategic plan lacked a structured investment plan. • Ignoring and not addressing the challenges with data could become a big threat. • Not properly addressing training will limit effective uptake and efficient usage of services. • Not properly addressing software innovation and development leads to inefficient use of resources. • There was a lack of evidence to make a case for increased funding. • The plan did not demonstrate the role of SNIC 2.0 in international organisations, in which it plans to

participate and influence. • There was a lack of monitoring metrics that could signal needs and threats.

Opportunities (all about added value) • Education and science is becoming increasingly global, and there is a strong emphasis within the EU on

developing an infrastructure for Open Science (EOSC). Thus, there are strong scientific and economic drivers for developing a national strategy on e-infrastructures, such as presented for SNIC 2.0.

• Improved support for users in SNIC 2.0 enables broadening of the user base to all science areas. • SNIC 2.0 has potential to form the core of a coherent national e-infrastructure and provide leadership in the

development of such an e-infrastructure. • Increased awareness among decision-makers of the importance of cost-effective e-infrastructure creates

community/political support for the e-infrastructure.


• URFI and the extended consortium for SNIC 2.0 broadens the sense of ownership (on many levels and from many aspects, such as URFI, other RIs and research communities).

• By exercising leadership, SNIC 2.0 can be a driver of e-infrastructure and e-science in the scientific community.

• By developing clear metrics for cost-effectiveness, SNIC 2.0 can increase understanding for the rapidly increasing need for e-infrastructure.

• SNIC 2.0 has the opportunity to explore cost-effectiveness, such as green energy solutions and professionalised processes.

• SNIC 2.0 and SRC could create incentives for professional code development.

Threats • Staying locked in the old SNIC model. • SNIC has a background in computing and supporting computationally skilled users, and may lack

competence and openness to support new user groups with a diverse set of needs. • Not broadening the scope (data, software) may make SNIC 2.0 obsolete and not a preferred partner. • URFI could lose trust in SNIC 2.0, leading to SNIC 2.0 losing funding. • If SNIC 2.0 does not step up and take national leadership, there is a chance that SNIC 2.0 will not be

successful. • The first procurement – and potential reduction of the number of universities where hardware is physically

located – with the new scheme may challenge the stability of the consortium. Expectation needs to be managed.

• If other RIs do not buy into SNIC 2.0 as a trusted partner, this could impede the successful implementation of the proposed strategic plan and budget.

Addressing the weaknesses and threats in the SWOT analysis can be used as a stepping stone for improving the strategic plan. Uppsala University (in close collaboration with URFI) has to become even more strategic and respond to the panel’s concerns by providing the SRC with a more detailed and worked-out implementation plan, budget (including investment plan) and risk analysis. Some areas that could be covered are funding cycles, prioritisations of services/clusters and who should supply them. It is also necessary to demonstrate how the transition from SNIC to SNIC 2.0 will be made smoothly. In this process, it is crucial to establish a governance structure of SNIC 2.0, not only at the level of the board, but also to formulate terms of reference for the board and for other advisory bodies.

Further measures to be strongly considered by the SRC During the course of its work, the panel identified some additional topics which did not fit into the themes and SWOT analysis above. These are all recommendations to the SRC, and the panel strongly encourages the council to consider them further.

In the future, the panel wanted to see a coherent national e-infrastructure that includes all digital services and e-infrastructures. To achieve this, the panel strongly recommended the SRC to write a national roadmap or plan for a coherent e-infrastructure, including not only HPC, but also all aspects of data, including archiving of data. Such an e-infrastructure should be able to support the transition towards open science. In this work, URFI will be an important partner, and the SRC should continue to build on this successful collaboration.

The panel questioned whether having a single host university for the national e-infrastructure is optimal, as this may not create sufficient ownership across universities. Other possibilities, such as a non-profit company owned by the universities as key stakeholders (Dutch model), or the Norwegian model, could be explored by the SRC. Even if this may not be on the agenda for the coming funding decision, it is something that could be considered for the future.

The panel recommended the SRC to continue exploring options for long-term sustainable funding of e-infrastructures. These type of infrastructures pose several challenges to funders; one aspect being that the demands for services increase much more rapidly than the funding. Another aspect is that when e-infrastructures take on responsibilities to share and disseminate data, the funding horizon needs to be very long.


Funding adequate e-infrastructures is a way of getting optimal return on investments already made/planned in data-producing RIs (such as Max IV and SciLifeLab).

The SRC should consider if they want to promote user fees as a way of increasing funding for SNIC 2.0, and to develop an awareness among users of the true costs. The SRC should also consider the possibility of including costs for e-infrastructures in project grants.

Final words The panel recognised this review as a strategic moment for all stakeholders, and an opportunity to develop the old SNIC into a new organization that is well-suited to support the broader Swedish science communities and has great added value. E-infrastructures constitute a fast-changing environment that evolves continuously, and SNIC is an organisation that is ready to step up to this challenge. Sweden has internationally well-recognized experts in the field, and this could be used as an advantage also internationally.

Västra Järnvägsgatan 3 | Box 1035 | 101 38 Stockholm | Tel 08-546 44 000 | [email protected] | www.vr.se

Vetenskapsrådet har en ledande roll för att utveckla svensk forskning av högsta vetenskapliga kvalitet och bidrar därmed till

samhällets utveckling. Utöver finansiering av forskning är myndigheten rådgivare till regeringen i forskningsrelaterade frågor

och deltar aktivt i debatten för att skapa förståelse för den långsiktiga nyttan av forskningen.

This report is the result of an independent analysis of a preliminary strategic plan and budget for the Swedish National Infrastructure for Computing (SNIC). The analysis was carried out by an advisory panel and will be an important contribution to the future development of SNIC.

international advisory review of the swedish ... - vr.se · were erik fledderus (director at surf,...

Documents