cloud for research and innovation - uk usa hpc workshop, oxford, july 205
TRANSCRIPT
![Page 1: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/1.jpg)
15/04/2023
Cloud for Research and Innovation - UK USA HPC Workshop July 2015 1
Cloud for Research and InnovationMartin Hamilton
Photo CC BY Flickr user atosorigin
![Page 2: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/2.jpg)
Cloud for research and innovation
1. Cloud adoption in enterprise IT– Trends in UK higher education– Bimodal IT versus bipolar institutions
2. Public clouds are starting to look more like HPC clusters– Orchestration, Infiniband, GPU and FPGA availability (at a price)
3. Code re-use and app portability– Will Docker and OpenStack be our Rosetta Stone(s)?– What does a unified computing model look like?
![Page 3: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/3.jpg)
15/04/2023
Cloud for Research and Innovation - UK USA HPC Workshop July 2015 3
1. Cloud adoptionin enterprise IT
![Page 4: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/4.jpg)
Cloud in UK higher education
Collaboration suites: Google Apps for Education» The University of Westminster estimates that it saved £1m by moving to
Google Apps for Education for email and other online collaboration facilities. They also state that “an additional benefit has been the reduced time spent in systems and user support with a minimal number of calls for support for such a significant system”, and that moving to the cloud has “liberated staff and students from the smaller storage limits of the previous in-house solution.”
http://services.google.com/fh/files/misc/univofwestminster.pdf
» Ealing, Hammersmith and West London College have also migrated to Google Apps. They observe that “each year, 20,000 new accounts are created providing 600 terabytes (around 150,000 DVDs worth) of storage in less than five minutes.” Subsequently to this article being written, Google moved to offering free unlimited storage.
http://www.wlc.ac.uk/college/news/article.asp?newsID=4228
![Page 5: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/5.jpg)
Cloud in UK higher educationCollaboration suites: Microsoft Office365» The University of the West of London moved its 14,000 students to Office365, and
observes that once a student graduates “they can maintain that Office365 account through the University for life. We can continue to supply them with information about the University, help them find their second job, their third job and so on. This isn’t just about collaboration while studying, it’s about creating a life-long connection between the students and the University.”
http://www.slideshare.net/Microsofteduk/university-of-west-london-case-study-office-365
» The University of Dundee migrated all of its 22,000 student accounts to Microsoft’s Office365 for Education service over a week. They note that this was achieved with zero down time, and estimate that Office365 “will save us at least £500,000 in infrastructure and staffing over five years.”
http://www.slideshare.net/Microsofteduk/university-of-dundee-case-study-office-365
![Page 6: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/6.jpg)
Cloud in UK higher education
Jisc Amazon cloud portal:› Monthly invoicing - credit cards are no
longer required for payment
› Itemized billing - consolidated across users/departments
› Billing in GBP, not dollars
› Setting of budget limits for individual user accounts or departments
› The retrieval of service usage information within own areas of responsibility
› Volume-discounts through aggregation across multiple educational institutions
![Page 7: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/7.jpg)
Cloud in UK higher education
Leveraging the Janet network:
![Page 8: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/8.jpg)
Cloud in UK higher education
Private peerings:
![Page 9: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/9.jpg)
Cloud in UK higher education
Private peerings:
![Page 11: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/11.jpg)
Cloud in UK higher education
Dogfooding:› We moved
our own core infrastructure to AWS
› Estimate that this saved £100K p.a.
› Cloud first / mobile first for new stuff
› NB we are not an HPC provider
![Page 12: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/12.jpg)
15/04/2023
Cloud for Research and Innovation - UK USA HPC Workshop July 2015 12
2. Public clouds arestarting to look like HPC clusters
![Page 13: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/13.jpg)
Case study: Azure HPC
![Page 14: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/14.jpg)
Case study: Azure HPC
![Page 15: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/15.jpg)
Case study: Azure HPC
![Page 16: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/16.jpg)
Case study: Azure HPC
http://azure.microsoft.com/blog/2015/07/09/azure-linux-rdma-hpc-available/
![Page 17: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/17.jpg)
Case study: Azure HPC
![Page 18: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/18.jpg)
Case study: Azure HPC
![Page 19: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/19.jpg)
Case study: Azure HPC
![Page 20: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/20.jpg)
Case study: Azure HPC
![Page 21: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/21.jpg)
Case study: Google Compute Engine
![Page 22: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/22.jpg)
Case study: AWS
![Page 23: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/23.jpg)
Case study: AWS
![Page 24: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/24.jpg)
Caveat Emptor: Magellan
http://science.energy.gov/~/media/ascr/pdf/program-documents/docs/Magellan_Final_Report.pdf
![Page 25: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/25.jpg)
Caveat Emptor: Magellan
http://science.energy.gov/~/media/ascr/pdf/program-documents/docs/Magellan_Final_Report.pdf
![Page 26: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/26.jpg)
15/04/2023
Cloud for Research and Innovation - UK USA HPC Workshop July 2015 26
3. Code re-use &application portability
![Page 27: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/27.jpg)
Re-use & portability: OpenStack
http://www.datacenterknowledge.com/archives/2014/11/07/cerns-openstack-cloud-to-reach-150000-cores-by-2015/
![Page 28: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/28.jpg)
Re-use & portability: OpenStack
http://www.datacenterknowledge.com/archives/2014/11/07/cerns-openstack-cloud-to-reach-150000-cores-by-2015/
http://www.datanami.com/2013/09/23/cern_turns_to_google_for_datacenter_direction/
![Page 29: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/29.jpg)
Re-use & portability: OpenStack
![Page 30: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/30.jpg)
Re-use & portability: OpenStack
![Page 31: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/31.jpg)
Re-use & portability: Containers
![Page 32: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/32.jpg)
Re-use & portability: Containers
https://wiki.openstack.org/wiki/Docker
![Page 33: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/33.jpg)
Re-use & portability: Performance
http://en.community.dell.com/techcenter/high-performance-computing/b/general_hpc/archive/2014/11/04/containers-docker-virtual-machines-and-hpc
![Page 34: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/34.jpg)
Re-use & portability: Open Container Project
![Page 35: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/35.jpg)
Re-use & app portability: Kubernetes & Heat
http://kubernetes.io http://techs.enovance.com/7104/multi-tenant-docker-with-openstack-heat
![Page 36: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/36.jpg)
Re-use & app portability: A unified model?
This is one model. Other models are available › Run OpenStack on your in-house facility [Q: Still need one?]
– This creates your own private/community cloud– Machine images can be programmatically converted to/from QCOW2
› Package your app with Docker and you can run it anywhere– Embraced by public cloud providers, can run in-house too as open source– Could be bare metal or on top of e.g. OpenStack [Q: Performance?]
› Lots of options for orchestration from general purpose uses– Kubernetes scheduling particularly interesting from an HPC perspective
› Open questions from an e-Infrastructure perspective:– Integrating identity management, authentication, authorization, accounting– User driven (instant gratification) versus controlled environment– Funding model for use of public and community clouds, potential for national deals– Lots of new stuff to learn about! (for developer, sysadmin, …)– That’s all fine for the compute, but what about the data?
Photo CC-BY Flickruser codep8
![Page 37: Cloud for Research and Innovation - UK USA HPC workshop, Oxford, July 205](https://reader035.vdocument.in/reader035/viewer/2022062711/55c264bfbb61ebd64a8b46d7/html5/thumbnails/37.jpg)
15/04/2023
Cloud for Research and Innovation - UK USA HPC Workshop July 2015 37
That’s all folks…
Except where otherwise noted, this work is licensed under CC-BY
Martin HamiltonFuturist, Jisc, London