Download - Joint techs keynote january
NREN strategies for Data-Intensive Science in a Carbon Constrained
World
Bill St. [email protected]
Unless otherwise noted all material in this slide deck may be reproduced, modified or distributed without prior permission of the author
Theme of this talk
• We have already lost the battle to save the planet from extreme climate change. Rather than focusing on reducing energy consumption, (Mitigation) we now need to focus on surviving climate change (Adaptation)
• Explosion of data and energy consumption by computers and networks is contributing to energy demand and CO2 emissions
• But big data and science will be critical as move to focus on adapting to climate change
• How can Internet and IT help us build NRENs and support science and
education that can adapt to global warming?
Changing NRENnetworking environment
• Global Virtual Research Communities
• Increasing co-operation between public and private researchers
• Rapidly changing users demands
• Increasing potential of commercial ICT-service providers
• Education: any time, any place, any device
• Citizen Science and M2M communications and sensors
• The disappearing campus IT & diminishing expertise in ICT centres of connected institutions
Half of US experienced record droughts or deluges in 2011
2010 warmest year ever – we are only at the start of the curve of the hockey stick. The worst is yet to come
6
Blame it on CanadaHow warming in the Arctic affects weather in Louisiana
• Warming Arctic slowing down jet stream • Basic Thermodynamics - polar temperatures
drive the jet stream, – There’s been a 20 percent drop in the zonal wind
speeds.• As get stream slows down, it leads to those
bigger kinks in the jet stream.– That amplification is associated with persistent
weather patterns that lead to “extremes” like drought, flooding and heat waves.
• Those slow-moving, persistent waves of weather energy may have played a role in the big snows that hammered parts of the West last winter, as well as some of the extreme winter weather that hit South West US and Europe
• http://summitcountyvoice.com/2012/01/14/global-warming-revenge-of-the-atmosphere/
Climate Forecasts
MIT
• MIT report predicts median temperature forecast of 5.2°C– 11°C increase in Northern Canada
& Europe– http://globalchange.mit.edu/pubs/abstract.p
hp?publication_id=990
• Last Ice age average global temperature was 5-6°C cooler than today– Most of Canada & Europe was
under 2-3 km ice
• Nearly 90 per cent of new scientific findings reveal global climate disruption to be worse, and progressing more rapidly, than expected.• http://www.skepticalscience.com/pics/Freud
enburg_2010_ASC.pdf
Future Droughts• Palmer Drought Severity Index, or PDSI.
• The most severe drought in recent history, in the Sahel region of western Africa in the 1970s, had a PDSI of -3 or -4.
• By 2030 Western USA could see -4 to -6. Drought in Texas clearly caused by global warming: http://goo.gl/QjHRS
• By 2100 some parts of the U.S. and Latin America could see -8 to -10 PDSI, while Mediterranean areas could see drought in the -15 or -20 range.
http://www.msnbc.msn.com/id/39741525/ns/us_news-environment
/
Dramatic changes in precipitation
• Every continent has suffered record rainfalls• Rains submerged one-fifth of Pakistan, a
thousand-year deluge swamped Nashville and storms just north of Rio caused the deadliest landslides Brazil has ever seen.
• Observed increase in precipitation in the last few decades has been due in large part to a disproportionate increase in heavy and extreme precipitation rates which are exceeding predictions made in models
New Challenge: Climate Adaptation
• Obama’s National Science Advisor John Holdren “Mitigation alone won’t work, because the climate is already changing, we’re already experiencing impacts….A mitigation only strategy would be insanity,”
• Equal emphasis given to adaptation – avoiding the unmanageable, and adaptation – managing the unavoidable.”
• Obama’s Climate Adaptation Executive Order– http://www.stumbleupon.com/su/1tU8go/www.good.is/post/obama-s-secret-climate-adaptation-plan/
Climate Change Impact on Internet and NRENs
• UK Government study Climate Change could ruin the Internet– http://www.grist.org/list/2011-05-09-climate-change-could-ruin-the-internet
• California aims to have 30% renewable power– Impact on reliability of power systems
• Last year Nuclear power plants in France were forced to shut down because cooling water was too warm
• Germany is committed to shutting down all of its nuclear plants
• Droughts will restrict production of hydro-electric power
• Energy shortages and disruptions are predicted to increase in the coming years
Impact on ICT sector
www.smart2020.org
According to IEA ICT will represent 40% of all energy consumption by 2030
ICT represent 8% of global electricity consumption
Future Broadband- Internet alone is expected to consume 5% of all electricity http://www.ee.unimelb.edu.au/people/rst/talks/files/Tucker_Green_Plenary.pdf
R&E biggest consumer!!
Australian Computer Society Studyhttp://www.acs.org.au/attachments/ICFACSV4100412.pdf
Per employee Per sector
The ICT energy consumption inhigher- ed
• Campus computing 20-40% electrical energy consumption on most campuses– Studies in UK and The Netherlands– http://goo.gl/k9Kib
• Closet clusters represent up to 15% of electrical consumption– http://isis.sauder.ubc.ca/research/clean-technology-and-energy/green-it/
• Campus data center alone represents 8-20% of electrical consumption– http://www.iisd.org/publications/pub.aspx?pno=1341
• IISD study demonstrated that moving Canadian research to cloud would pay for itself in energy savings and CO2 reduction– http://www.iisd.org/publications/pub.aspx?pno=1341
The real cost of campus computing
• Land - 2%• Core and shell costs – 9% • Architectural – 7%• Mechanical/Electrical – 82%
– 16% increase/year since 2004Source: Christian Belady
Belady, C., “In the Data Center, Power and Cooling Costs More than IT Equipment it Supports”, Electronics Cooling Magazine (February 2007)
The Data Deluge
Genomic sequencing output x2 every 9 month
Climate model intercomparisonproject (CMIP) of the IPCC
2004: 36 TB
2012: 2,300 TB
1330 molec. bio databases Nucleic Acids Research (96 in Jan 2001)
MACHO et al.: 1 TBPalomar: 3 TB2MASS: 10 TBGALEX: 30 TBSloan: 40 TB
Pan-STARRS: 40,000 TB
Source: Ian Foster, UoChicago
Big science has achieved big successes
LIGO: 1 PB data in last science run, distributed worldwide
ESG: 1.2 PB climate datadelivered to 23,000 users; 600+ pubs
OSG: 1.4M CPU-hours/day, >90 sites, >3000 users, >260 pubs in 2010
Robust production solutionsSubstantial teams and expenseSustained, multi-year effortApplication-specific solutions, built on common technology
Source: Ian Foster, UoChicago
But small science is struggling
More data, more complex dataAd-hoc solutionsInadequate software, hardwareData plan mandates
Source: Ian Foster, UoChicago
19
Growth in sensor networks and Citizen Science
Real Time Health Monitoring
Glacier Tracking
Smart Trash
THE CHALLENGEWe need solutions to address climate change, data deluge, needs of scientists, global collaboration, the evolving network of any time, any place, any device and yet addresses the challenge of disappearing IT on campus while still providing a leadership role in next generation Internet and broadband, and find ways to pay for it all in an era of severe fiscal constraint.
21
THE SOLUTION
1. Brokered Green Clouds and off site campus IT2. Software Defined Networks (OpenFlow)3. NREN national wireless network 4. Global Interconnected Dynamic Optical Networks5. eScience Platforms with next gen IdM6. Community anchor IXPs with CDN and M2M hosting7. New billion dollar revolving green energy funds at many
universities
Universities moving to eliminate IT departments
• Already many primary functions of IT department are being outsourced to the cloud– E-mail, web, DNS, research computing, etc– University of Western Australia has outsourced virtually all campus servers to an
external private cloud
• Even routing, network and firewall functions being outsourced to NREN– AARnet, SUnet and other NRENs offering border gateway routing services with
collapsed IP backbones– Software Defined Networks makes it easy to configure outsourced LAN– Network facilities can be located
• Increasingly most traffic is in/out of campus, instead of within– Social networking, P2P, Clouds, Kuali, Blackboard– Future of Campus IT – high speed optical network connected to WiFi/5G hot
spots with tablets– No servers, no LAN
MIT to build zero carbon data center in Holyoke MA
• The data center will be managed and funded by the four main partners in the facility: the Massachusetts Institute of Technology, Cisco Systems, the University of Massachusetts and EMC.
• It will be a high-performance computing environment that will help expand the research and development capabilities of the companies and schools in Holyoke– http://www.greenercomputing.com/news/2009/06/11/ci
sco-emc-team-mit-launch-100m-green-data-center
NREN Brokered Cloud for IT departments and Researchers
• Internet 2 Net +– Provisioning of multi vendor cloud services leveraging the Internet2
Network and InCommon Federated Authentication – Interoperable marketplace for services where individual institutions
might procure services from a wide range of cloud services providers.
• HEFCE and JISC to Deliver Cloud-Based Services for UK Research
– Besides providing brokered cloud services they are also providing cloud “solutions” for IT departments and researchers
– http://www.hpcinthecloud.com/hpccloud/2011-06-27/hefce_and_jisc_to_deliver_cloud-based_services_for_uk_research.html?utm_medium=twitter&utm_source=twitterfeed
• SURFnet: Community Cloud Models and the Role of the R&E network as a broker for cloud services– http://www.slideshare.net/haroldteunissen/community-clouds-shared-infrastructure-as-a-service
OpenFlow Follow the wind/Follow the sun
Cloud Manager
Host Resource
Cloud Manager
Network Manager
VM
Mantychore2
Host Resource
Canadian GSN Domain
European GSN Domain
Dynamically ConfigureIP Tunnel
• Shudown VM• Copy Image • Update VM Context
• Start VM
Export VM
VMVM
Internet
Notify EU Cloud Manager
Cloud Proxy Host Cloud Proxy
Lightpath
Optical switch Optical switch
Shared storage
Shared storage
Host
OpenFlow-based cloud
Open Virtual Switch (OVS)
Host
eth0 eth1
Ethernet Switch
Open Virtual Switch (OVS)
Host
eth0 eth1Open Virtual Switch (OVS)
Host
VM VM
eth0 eth1Open Virtual Switch (OVS)
Host
VM
eth0 eth1
VM
OpenFlow Control plane
OpenFlow Data plane
OF Controller
OpenFlow Network B
OVS
eth0 eth1
Internet
eth0 eth1
OVS
VM VM VM VM
OpenFlow Network A
Green Clouds International• GreenLight explores how researchers can take advantage of data centers linked by
high-speed networking in an era of carbon-thrifty computing • Recent studies migrating virtual machines to green energy sites indicate that 100
Gb/s networks are far superior to 10 Gb/s to make this transparent.• SURFnet 7 lightpath connection to GreenQCloud in Iceland
SURFconecxt control of lightpath to GreenQCloud in Iceland
Future Global Network of Green Clouds interconnected by GLIF
Science Cloud Communication Services Network
• Enterprise clouds use commodity internet; computational clouds for data-intensive science require dynamic cloud provisioning integrated with dynamic high performance.
• TransCloud: example of dynamic networking & dynamic cloud provisioning
Example of working in the TransCloud[1] Build trans-continental applications spanning clouds:• Distributed query application based on Hadoop/Pig• Store archived Network trace data using HDFS• Query data using Pig over Hadoop clusters[2] Perform distributed query on TransCloud, which currently spans the following sites:• HP OpenCirrus• Northwestern OpenCloud• UC San Diego• Kaiserslautern
Source: Maxine Brown
Building a NREN wireless network
• Vision: to allow students, researchers and employees to collaborate, research, learn anytime and anywhere they seem fit!
• Also Internet of Things – Machine to Machine communications
• Existing 3G and 4G networks cannot handle data load– Roaming gateways prevent global seamless access– Voice centric architectures
• New mobile networks seamlessly integrate with WiFi on campus– New Wifi 2.0 standards 802.11u allow for data handoff from 3G
networks– Eduroam can be the global authorization tool– OpenFlow can be used to architect integrated solutions from wireless
node across optical network
Impact of NREN wireless networks
34
• The phone is a also a sensor platform
• Processing is done in real time in the cloud– Allowing processing that can’t be done on the device– Big data analysis
• New campus or hot spot centric architectures integrating LTE and Wifi– See SURFnet pilot
http://www.surfnet.nl/en/nieuws/Pages/Backgroundarticle.aspx
• WiFi nodes can be powered by renewable sources such as roof top solar panel over 400Hz power systems or ethernet power
The Regulatory Challenge• Today’s SIM-card locks user to the network
• If NREN becomes a MVNO with own SIM-cards, users could roam seamlessly around the globe
• Only public service providers have access to IMSI-numbers for SIM-cards
• One option is to lobby regulators to give R&E networks access to IMSI-numbers
39
Importance of GOLEs• Increasingly more research and education is international collaboration
– Cornell- Technion announcement– US overseas university campuses in UK and elsewhere– GOLES enable direct peering of regional networks or even institutions
• Many researchers need access to commercial clouds and data specialists– AUP issues often prevent NRENs from directly connecting up these institutions– Genomics and bio-informatics processing and climate modeling
• Many commercial research institutions need access to lightpaths– GOLES provide neutral access points for interconnect to AUP free lightpaths
• Enables new services– Software Defined Network using Switched lambdas
Virtual Organisations
NBIC
Group
Netherlands BioInformatics Centre (NBIC)
Apps.NBIC.nl
Supporting Services• SURFfederatie• SURFteams• OpenSocial
My
Exp
eri
men
t
Pu
bM
ed
Gri
d r
es.
Pu
blis
her
N=6 N=10 N=30GuestsN=20
N=66
Virtual IdPNetwork
ServicesStorage Services
Compute
Services
Network Broker Storage Broker Compute
Broker
Instrument
Services
Instrument Broker
Collaboration Infrastructure (SURFconext)
Generic Broker
AAI provisioning
groups
Attrib.
mgmt
…
Source: SURFnet
45
Extending science and education to the community
• Community anchor Internet Exchange Points help clear the bottleneck of content peering– Co-hosting of CDN caching boxes– Managed by NREN– Examples include KAREN (New Zealand), BCnet and UNINETT (Norway)
• Minimize tromboning of R&E traffic to homes and schools• Can support extension of Eduroam to community WiFi spots and/or community last
mile networks• Allows for M2M traffic and anywhere, anytime traffic to propagate through the
community
Community IXP managed by NREN
47
$1 billion funding program• Green revolving funds are either part of a university endowment program or publicly traded
entities.– http://www.sustainablebusiness.com/index.cfm/go/news.display/id/23028
• They make investments in energy efficiency and GHG reduction initiatives. Payback typically 32%
• ICT can represent up to 40% of the electrical energy consumption at university and growing
• The obvious low hanging fruit is to move, as much as possible the closet clusters and campus data center facilities to commercial clouds. Next is network infrastructure such as routing and servers
• Other obvious money saving practices are to power laptop and cell phone charging stations with roof top solar panels or micro windmills, deploy solar/wind powered WiFi nodes, and use on the move electric charging for campus utility vehicles, etc
• Campus IT folk and NRENs need to educate managers of such funds the IT and networking can play a much more significant role in reducing energy consumption and GHG emissions then traditional facilities based solutions
Cyber-infrastructure in a Carbon Constrained World
http://net.educause.edu/ir/library/pdf/ERM0960.pdf
Let’s Keep The Conversation Going
Blogspot
Bill St. Arnaudhttp://green-broadband.blogspot.com
http://twitter.com/BillStArnaud
E-mail list