Dr. B.B Prahlada Rao [email protected]
19 Oct 2013
Centre for Development of Advanced Computing
C-DAC Bangalore, India
2nd NKN Workshop @ IISc
Cloud Computing: Trends & Opportunities
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 1
Presentation Plan
• Introduction to Cloud computing o Evolution of Technologiees: Computing, Networking
o Definitions Cloud Computing
o Characteristics of Cloud Computing
• Scientific Clouds
• C-DAC Cloud Initiatives
• Government Clouds
• Clouds -IOT
• Sensor Clouds
• Conclusions
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 2
Definitions- Cloud Computing
• Cloud Comp is a style of computing, where massively scalable IT related capabilities are provided ‘as a service’ across the Internet to multiple external customers. ….. Gartner – It means Cloud should deliver a service on a subscription basis,
allow access from anywhere and provide economies of scale. – Cloud will go cross vendor and feature shared resources.
Q. Is Cloud a metaphor for the Internet? • Cloud computing is an evolution of the Internet computing or usage of the
Internet as a computing infrastructure & resource. CC is used to provide services such as Data Storage, Software Applications, and email and file exchanges
• CC is Ubiquitous network access
• CC promised infinite elastic computing resources.
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 4
The Next Revolution in IT?
Classical Computing
– Buy & Own
Hardware, SystemSW,
Applications: to meet peak needs.
– Install, Configure, Test,
Verify, Evaluate
– Manage
– ..
– Finally, use it
– $$$$....$(High CapEx)
Cloud Computing
– Subscribe
– Use It
– $ - Pay for what you use,
based on QoS
– No CapEx ,Only OpEx
Ev
ery
18
mo
nth
s?
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 8
New IT Trends
• Ownership -> Leasing Model
– Infrastructure, platform, applications
• PC -> Internet Data Center
– Building applications for a single user large
number of concurrent consumers
• Web 2.0 & Virtualisation
– Building 3rd party IT services by composing
multiple independent services
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 9
Grid Computing
Utility Computing
SaaS Computing Cloud Computing
Solving large
problems with
Parallel Comp.
Made
Main Stream
by Global
Alliance
Offer Comput.
resources as
Metered
Service
Introduced in
late 1990s
Network-based
subscriptions
to applications
Gained
momentum
in 2001
Next-Generation
Internet
computing
Next-Generation
Data Centers
Evolution of Cloud Computing
Ref No: 6 Trends Cloud Comp- NKN
WS 2013
Trends Cloud Comp-
NKN WS 2013 19 Oct 2013
Promised Benefits of Public Clouds
• No upfront infrastructure investment – No procuring hardware, setup, hosting, power, etc
• On demand access – Lease what you need and when you need
• Efficient Resource Allocation – Globally shared infrastructure, can always be kept busy by serving users
from different time zones/regions.
• Nice Pricing – Based on Usage, QoS, Supply and Demand, Loyalty
• Application Acceleration – Parallelism for large-scale data analysis, what-if scenarios studies
• Highly Availability, Scalable, and Energy Efficient
• Supports Creation of 3rd Party Services & Seamless offering – Builds on infrastructure and follows similar Business model as Cloud
11
Inside Cloud
Storage Provisioning
VM Provisioning
Network Provisioning
Pricing
Billing
Pricing
Accounting
Monitoring
Cloud Middleware
Interface
Virtual Resources
Physical Resources
User Corporate
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 12
Cloud Architecture
Virtual Machine (VM), VM Management and Deployment
QoS Negotiation, Admission Control, Pricing, SLA Management,
Monitoring, Execution Management, Metering, Accounting, Billing
Cloud programming: environments and tools Web 2.0 Interfaces, Mashups, Concurrent and Distributed Programming,
Workflows, Libraries, Scripting
Cloud Applications:
Social computing, Enterprise, ISV, Scientific, CDNs, ...
Ad
ap
tive M
an
ag
em
en
t
Core Middleware
User-Level Middleware
System level
User level
Au
ton
om
ic / Clo
ud
Eco
nom
y
Apps Hosting Platforms
Cloud resources
13 19 Oct 2013 Trends Cloud Comp- NKN WS 2013
Trends Cloud Comp- NKN WS 2013
Cloud Anatomy
Application Services (Services on Demand)
Gmail, GoogleCalender
Payroll, HR, CRM etc
Sugarm CRM, IBM Lotus Live
Platform Services (Resources on Demand)
Middleware, Integration, Messaging, Information, connectivity etc
AWS, IBM Virtual images, Boomi, CastIron, Google App engine
Infrastructure as services (Physical Assets)
IBM BlueHouse, VMWare, Amazon EC2, Microsoft Azure, Sun Parascale …
Three models based on the type of service offered to the user
19 Oct 2013 14
Cloud Deployment Models
Private/Enterprise Clouds
Cloud computing model run
within a company’s own Data Center / infrastructure for internal and/or partners use.
Public/Internet
Clouds
3rd party, multi-tenant Cloud
infrastructure & services:
* available on
subscription basis (pay as you go)
Hybrid/Mixed Clouds
Mixed usage of private and public
Clouds: Leasing public cloud services
when private cloud capacity is insufficient Trends Cloud Comp- NKN
WS 2013
Technology Components for Cloud Infrastructure
Technology Components :
• Virtualization
• Provisioning Automation and Scheduling
• SLA Management– Computing, Storage and Network latencies /bandwidths
• Billing and Charging
• Data-Center Power/ Cooling/ Space
Trends Cloud Comp- NKN WS 2013 19 Oct 2013 16
Traditional Arch. Vs Virtualization Arch.
Hardware
Operating System
App App App
Traditional Stack Hardware
App
OS
App App
Hypervisor
Virtualized Stack
OS OS
Virtual Architecture
• Many OS to one server
• Separation between SW and HW
• Many applications on one server
• Typical load on server is 80 –95%
• Dynamically optimized resources
Traditional Architecture
• One OS on one Server
• Tightly coupled HW and SW
• One Application on One Server
• Typical load on server is 5 –15 %
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 17
• Most Important Component.
• Thin Layer Software
• Allow Multiple OS’s to run Concurrently.
• Dynamically Partitioning and Sharing the physical resources such as CPU, storage, memory and I/O devices.
• Abstracts the Hardware from the VM’s.
Hypervisors
Scheduler
Hypercall
CPU Virt MMU Virt I/O Virt
Control Interface
Network Virt
Hypervisor Components
Interrupts
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 18
Cloud Middleware
Cloud Middleware A software used to integrate services, applications and content available on the cloud.
Key Features:
Management and monitoring Resource Scheduler Data management Identity / security management Service hosting, mediation and management User interfaces and portals Billing and metering
19 19 Oct 2013 Trends Cloud Comp- NKN WS 2013
Cloud Applications
•Scientific/Tech Applications •Business Applications •Consumer/Social Applications
Science and Technical Applications
Business Applications
Consumer/Social Applications 19 Oct 2013 Trends Cloud Comp- NKN WS 2013 20
Penguin Computing (POD) http://www.penguincomputing.com/services/hpc-cloud/pod
• Provides HPC on Demand
• Powered by Scyld Cloud Management Platform.
• Storage System is based on the distributed open-source storage system Ceph which supports file-based, block-based and object-based storage.
• Management of Virtual Servers leverages OpenStack (Open-Source solution) for creating and managing virtual servers in a cloud environment
Benefits:
• Ease of use: Compute resources are available immediately
• True HPC technology Including GPUs, low-latency interconnects, and a non-virtualized compute stack
• Multiple PODs: Different environments for different industry needs
19 Oct 2013 22 Trends Cloud Comp- NKN WS 2013
OSDC (Open Science Data Cloud)
OSDC Provides : • Petabyte-scale Data Cloud infrastructure • Comput’l resources & services for scientists to analyse , manage, and share data • Platform for data intensive science so that new types of data intensive algorithms can be
Developed Tested and used over large amounts of heterogeneous scientific data .
OSDC Projects : • Bionimbus - collaboration between the Institute for Genomics and Systems Biology
(IGSB) and the OSDC to develop open source technology for managing, analysing, transporting, and sharing large genomics datasets in a secure and compliant fashion.
• Conte Cloud - is cloud computing infrastructure designed to store genomics, electronic medical records, and other sensitive data in a secure and compliant environment. – The mission is to apply integrated informatics and mathematical modeling to predict
genetic and environmental factors underlying mental health and illness .
• Bookworm - Offers a way to interact with digitized book content and full text search . Bookworm uses ngrams extracted from books, in the public domain and integrates library metadata, including genre, author information, publication place and date.
• Matsu: open source technology for cloud-based processing of satellite imagery to support the earth sciences.
19 Oct 2013 23 Trends Cloud Comp- NKN WS 2013
AWS -Amazon Web services
• Amazon Elastic Compute Cloud (EC2): offers two cluster instance types: – cluster compute instances and
– cluster graphics processing unit (GPU) instances.
– Provides scalable virtual private servers using xen hypervisor.
• Amazon Simple Storage Service (S3): – Provides Web Service based storage.
• Amazon Elastic MapReduce (EMR): Allows processing of vast amounts of data using a hosted Hadoop framework running on the web-scale infrastructure of EC2 and Amazon S3
19 Oct 2013 24 Trends Cloud Comp- NKN WS 2013
Windows Azure
• Cloud computing platform and infrastructure by Microsoft for building, deploying and managing applications and services .
SERVICES :
• Virtual machines(IAAS) - Virtual machines let developers migrate applications and infrastructure without changing existing code, and can run both Windows Server and Linux virtual machines .
• PAAS : Customers can create web sites in PHP, .NET and Node.js, or select from several open source applications from a gallery to deploy.
• Windows Azure can be used to build a web application that runs and stores its data in Microsoft datacenters.
• Hadoop services through HDInsight: Deploys and provisions Apache Hadoop clusters in the cloud, providing a software framework designed to manage, analyze and report on big data.
• Various database services for data management .
19 Oct 2013 25 Trends Cloud Comp- NKN WS 2013
Google Cloud Platform
• Google Cloud Storage : online file storage web service for storing and accessing your data
on Google's infrastructure. The service combines the performance and scalability of Google's cloud with advanced security and sharing capabilities. It is an Infrastructure as a Service comparable to Amazon S3 online storage service .
• BigQuery : Enables interactive analysis of massively large datasets working in conjunction with Google Storage . It is an Infrastructure as a Service that may be used complementarily with MapReduce. To use the data in BigQuery, it first must be uploaded to Google Storage .
• Google Compute Engine(IAAS) : Google Compute Engine uses KVM as the hypervisor, and only supports guest images running Linux . Every Google Compute Engine instance starts with a disk resource. Depending on the selected machine type, the instance may start with scratch disk space, persistent disk space, or both .
• Google Cloud SQL : Mysql on the cloud Google Cloud SQL is a web service that allows you to create, configure, and use relational databases that live in Google's cloud . It is Fully managed and No worrying required about tasks such as replication, patch management, or backups. These are all taken care of.
• http://en.wikipedia.org/wiki/BigQuery • http://en.wikipedia.org/wiki/Google_Storage
19 Oct 2013 26 Trends Cloud Comp- NKN WS 2013
Future Grid
• FutureGrid Project supports several clouds, distributed among five sites, in aggregate providing the capacity of over a thousand cores.
• FutureGrid clouds are configured with Nimbus, OpenStack and Eucalyptus, which support interfaces that are roughly compatible with AWS EC2/S3, allowing users to move between clouds relatively easily.
• FutureGrid testbed provides capabilities that allow users to experiment with open-source cloud middleware and virtualization platforms .
• FutureGrid Project Makes it possible for researchers to tackle complex research challenges in computer science related to the use and security of grids and clouds. These include topics ranging from authentication, authorization, scheduling, virtualization, middleware design, interface design, and cybersecurity, to optimization of grid-enabled and cloud-enabled computational schemes for researchers in astronomy, chemistry, biology, engineering, atmospheric science and epidemiology.
• FutureGrid provides Hadoop platform and the iterative map/reduce Twister plaftorm to allow users to experiment with MapReduce applications and middleware .
Ref: • http://scienceclouds.org/infrastructure-clouds/ • http://scienceclouds.org/infrastructure-clouds/futuregrid/
19 Oct 2013 27 Trends Cloud Comp- NKN WS 2013
RED Cloud
Red Cloud is a subscription-based cloud computing service for research and education.
Red Cloud services : • Red Cloud is an Infrastructure as a Service (IaaS) that runs Eucalyptus . Subscribers have root
access to virtual servers and virtual disks. • Red Cloud with MATLAB” is a Software as a Service (SaaS) that runs MATLAB Distributed
Computing Server and features NVIDIA GPUs. Subscribers program applications on their desktops using their licensed copy of Parallel Computing Toolbox™ and then scale up to Red Cloud with MATLAB using MATLAB Distributed Computing Server.
• The user selects cac scheduler as the MATLAB parallel scheduler. This will cause parallel/distributed jobs to be submitted to Red Cloud with MATLAB for execution
• Both “Red Cloud” and “Red Cloud with MATLAB” are available to researchers at other academicinstitutions.
• Industry access is limited to “Red Cloud” as an IaaS . Ref: • http://www.cac.cornell.edu/redcloud/ • http://www.cac.cornell.edu/wiki/index.php?title=Red_Cloud_with_MATLAB • http://www.cac.cornell.edu/RedCloud/FAQ.aspx
19 Oct 2013 28 Trends Cloud Comp- NKN WS 2013
Stratus http://www.cloudbus.org/ccgrid2012/posters/abstract_stratus_pelle_jakovits.pdf
• Stratus is a new framework based on the BSP model (Bulk Synchronous Parallel)
• Developed at University of Tartu as part of the Scientific Computing in the Cloud (SciCloud) project.
• Bulk Synchronous Parallel (BSP) is a distributed computing model for iterative algorithms that are very often used in scientific simulations.
• Stratus framework: is to provide a Distributed computing platform and tools for performing large scale Scientific computing simulations and experiments.
19 Oct 2013 29 Trends Cloud Comp- NKN WS 2013
StratusLab ● StratusLab Toolkit: open source cloud distribution Customized
Environments Current version StratusLab is 1.4 ● StratusLab project is to create a private cloud distribution that
permits resource center administrators to deploy grid services over the cloud’s virtualized resources
● StratusLab is aimed at service provisioning, networking and research of cloud and virtualization technologies to simplify and optimize the existing distributed computing infrastructures like the European Grid Infrastructure (EGI).
Grid Services on the Cloud:Grid services Deploying over virtualized
resources would allow easy (re)deployment even if some node crashes.
Customized Environments : Individual users are also likely to require customized environments containing, eg, their own proprietary software.the ability to create virtual machines
Provision of Common Appliances: Providing simple, stock images of common operating systems lowers the barrier to creating customized images as well as improves the utility of the cloud infrastructure
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 30
Scientific Cloud: HPC as a Service
• Cluster as an Infrastructure Service (IaaS) • Provisioning of on demand HPC clusters • Provisioning of Map reduce clusters to process large data sets • Support for MPI as a tightly-coupled applications category. • Design methodology for providing/supporting installation of MPI
libraries in multiple machines simultaneously • Storage as a Service (StaaS): CloudVault • SaaS:
– Job Submission Portal (C_JSP) , Cloud Access Portals – Monitoring tools
• PaaS: Program development and analysis Tools • Applications as Service (AaaS) PSEs
– PSE for Climate Modelling – PSE’s for Bio Informatics
• Types of applications supported – Sequential & Concurrent Applications – Data Parallel Applications - Map Reduce
– Parallel Programs: OpenMP ,Pthreads, MPI
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 31
Nimbus: set of open source tools
Mission:
To evolve the infrastructure with emphasis on the needs of science
• Provides Infrastructure-as-a-Service (IaaS) Cloud Comp solution. Features :
– Open Source IaaS – Storage Cloud Service – Remote Deployment & lifecycle management of VMs – Compatibility with Amazon EC2 & S3 – Support for X509 credentials – Easy to use cloud client – Fast Propagation (LAN torrent) – Per user quota allocation – Easy user management – One click clusters (contextualization) – VM network configuration – Local resource management plugin
32 19 Oct 2013 Trends Cloud Comp- NKN WS 2013
• GridGain is a software middleware that enables development of
HPC& data intensive distributed applications for real-time Big Data
processing
• Java based
• Open source
GridGain
33 19 Oct 2013 Trends Cloud Comp- NKN WS 2013
Openstack Features
• NASA & Rackspace (Open source-2010), 3386+ people & 183+ Companies all over the world
• Leverage Resource pools of on-demand • Focuses HPC deployments • Dashboard to allows cloud
administrators to create & manage projects
• Simultaneously launch and manage multiple resource instances
• Users can launch and customize instances in the OpenStack Dashboard within the limits their administrators have set.
34 19 Oct 2013 Trends Cloud Comp- NKN WS 2013
• Open source Tech to Build Ubiquitous public & private cloud computing
platform
• Open source, Open design ,Open development, Open community
Open Cirrus • Open Cirrus is an open cloud-computing research testbed.
• It is shared : Researches , applications , infrastructure, data sets.
• Global services: sign on, monitoring, store. Open source stack (prs, tashi, hadoop).
• Sponsored by HP, Intel, and Yahoo! (additional support from NSF).
• 15 sites currently, target of around 20 in the next two years.
35 19 Oct 2013 Trends Cloud Comp- NKN WS 2013
Services Provided By Open Cirrus Sites • PRS Service : The lowest level service is based on the notion of a physical
resource set (PRS).
• Cluster Management Services : It runs several different cluster management services like CaaS , Tashi.
• Application Framework Service: Open Cirrus sites also provide higher level services, such as Hadoop, Pig, and MPI, that support user-level applications and services.
C-DAC Cloud Initiatives
• Pan C-DAC Cloud Computing-Initiative: – Meghdooth: Open Source Cloud Stack – Scientific Cloud Computing: Scientific Cloud Stack, IaaS, StaaS, PaaS
– E-Sikshak: C-DAC’s E-Learning on Cloud – MANTRA: MAchiNe Assisted TRAnslation System over CLOUD
– Disaster Recovery: (DR) solution on Cloud – eSanjeevani : Web based Telemedicine Appn, CDAC Mohali.
– Megh Shushrut: C-DAC Hospital Mgmt Information System
• Pan C-DAC Cloud Infrastructure • Nano Computing Clouds
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 36
Meghdoot Stack
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 37
• One stop solution for implementing Cloud environment.
• One single middleware bundle that incorporates all pre-requisites to establish a complete functional Cloud
• Provision to offer IaaS, PaaS, SaaS
• Free and Open source
• Based on Open standards • Compliance to cloud
recommendations by International bodies.
C-DAC Scientific Cloud
Scientific Cloud is a model for on-demand access to a shared pool of HPC resources (e.g.,networks, servers, storage, applications) that can be easily provisioned as and when needed by the researchers and scientists.
Benefits of Scientific Cloud
• Easy access to large infrastructures
• On demand access to HPC resources
• Virtual ownership of cloud resources
• Ease of deployment
• Scientific Cloud provide an alternative for HPC outreach
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 38
Scientific Cloud: HPC as a Service
• Cluster as an Infrastructure Service (IaaS) • Provisioning of on demand HPC clusters • Provisioning of Map reduce clusters to process large data sets • Support for MPI as a tightly-coupled applications category. • Design methodology for providing/supporting installation of MPI
libraries in multiple machines simultaneously • Storage as a Service (StaaS): CloudVault • SaaS:
– Job Submission Portal (C_JSP) , Cloud Access Portals – Monitoring tools
• PaaS: Program development and analysis Tools • Applications as Service (AaaS) PSEs
– PSE for Climate Modelling – PSE’s for Bio Informatics
• Types of applications supported – Sequential & Concurrent Applications – Data Parallel Applications - Map Reduce
– Parallel Programs: OpenMP ,Pthreads, MPI
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 39
SciCloud Service Offerings
19 Oct 2013 Trends Cloud Comp- NKN WS 2013
GUI
IaaS PaaS
SaaS
Cloud Portal ( Selects the Resource)
Virtual Machine (OS, Time, Type)
Virtual Cluster
(No. of nodes, type, Time)
Hadoop Cluster
(No. of nodes, Time)
MPI Cluster
(No. of nodes, Time)
Cloud Vault (StaaS)
Web interface
Small
Medium
Large
PSEs for
Bioinformatics Climate Modelling
Job Submission
Portal
41
Cloud Vault
End-User
Desktop Application
Web Application
Mobile Application
Swift
Storage and Data Center
Physical Storage
OpenStack Object Storage
Cloud Vault is your personal hard drive in the
cloud. Store any kind of file like Documents, Music, Videos, Photos, and Virtual Machine
Images on CDAC Cloud Vault secure servers.
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 42
CloudVault: C-DAC Cloud StaaS- Features
• C-DAC Cloud Vault: Storage As a Service.
• Organizations can use CloudVault to store Large data efficiently, safely, and cheaply.
• Capable of storing billions of Storage objects across Distributed Storage Servers.
• Extremely Scalable in terms of Storage Size (Several Petabytes) and Storage Capacity (Number of objects).
• User can Upload, Download, file or Directory irrespective of Size/Type.
• Users can access CDAC CloudVault via GUI or Command Line or through APIs.
• Similar to Amazon Web Services (S3).
19 Oct 2013 43 Trends Cloud Comp- NKN WS 2013
Dissemination: Workshop(s) / Seminar(s) Training
• Prahlada Rao B.B., Payal Saluja, Vineeth Simon Arackle, Half-Day Tutorial “Cloud Computing for HPC” C-DAC’s National Conference on Parallel Computing Technologies (ParCompTech2013), NIAS, Bangalore, 21-23rd Feb 2013.
• Prahlada Rao B.B, Payal Saluja, Neetu Sharma, Ankit Mittal, Shivay Veer Sharma, “Cloud Computing for IOT & Sensing Based Applicationss”, in IEEE International Conference on Sensing Technologies, ICST 2012, Kollkata, India, Dec 2012.
• Prahlada Rao B.B., Payal Saluja, Vineeth Simon Arackle, Half-Day Tutorial “Open Source Platforms for Scientific Clouds” CSI’s Annual Conference on Advances in Cloud Computing (ACC2012), 26th July 2012, Bangalore.(60+ participants from Industry, Academia)
• Payal Saluja, Prahlada Rao, Ankit Mittal, Rameez Ahmad, “CDAC Scientific Cloud: On Demand Provisioning of Resources for Scientific Applications”, in Proceedings of 18th International Conference PDPTA 2012, Las vegas, Nevada, USA, 16-19 July 2012.
• Prahlada Rao B.B., Given a Tutorial On “Cloud Computing”, International Conference on Trends in Knowledge and Information Dynamics, ICTK 2012, organized by ISI, NIMHANS Conv Centre, Bangalore, India, 10 July 2012.(150 participants from Industry, Academia)
• Dr. Prahlada Rao BB, Cloud Computing for Telemedicine, National Workshop on Telemedicine Today & Tomorrow, C-DAC Mohali, An Invited Talk, 6th July 2012.
• Prahlada Rao B. B. , National Workshop on Cloud Computing, conducted at CDAC Knowledge Park, Bangalore, 21st May 2011 .(80+ participants from Industry, Academia)
• Vineeth Simon Arackal, Arunachalam B, Payal Saluja, Prahlada Rao B B, Co-existence of Cloud and Grid: A Case Study in Service Oriented Grid GARUDA. The International Conference on Information Technology- New Generations (ITNG 2011), Las Vegas, USA. 11-13April 2011.
• Vineeth Simon Arackel, “ Coexistence of Grid and Cloud” at "Grid To Cloud (G2C)- Challenges and Applications" Workshop at VIT, Vellore, 2nd April 2011.
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 44
Cloud Vault: Features • Reliable Storage 24x7 • Storage Access through Internet :
• Requested MB/GB Storage • Required duration
• Interfaces GUI - Access cloud vault files
• File /Directory Operations : – Create Directory – Files /Directories Listing – File/ Directories Upload/ Download – File Editing – Files Synchronization
Java APIs • Authentication/Login
• File Operations – Listing – File upload – File Download
Functionality: • Virtual image storage (used for provisioning
of IaaS)
• Storage of any type of file including mpeg,
jpeg, videos
• Unstructured Data Storage (NO-SQL
database)
• Backup, Archive/ Retention
• Scientific Databases in the cloud
Technologies & Softwares:
– Openstack Swift Storage
– GlusterFS
– Cyberduck
– Rackspace Storage APIs
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 45
SuMeghaLab: Cloud Lab Kit
Components : Hypervisor
Xen
Cloud Middleware Nimbus, Openstack Swift
Storage Glusterfs
Portal SuMegha portal Job Submission Portal Cloud Vault Portal
PSE PSE for SFM PSE for NGS Pipeline
Supported Images:
Virtual Machines
CentOS (small, large, medium)
Job Submission Portal
Virtual Clusters
MPICH enabled with job submission portal
HADOOP enabled
PSE enabled images
SFM
NGS Pipeline
19 Oct 2013 47 Trends Cloud Comp- NKN WS 2013
• Packaged Software to Create automated deployment of Private Clouds. • Suitable for Academic institutes & Research organizations to setup cloud
labs, by users having no expertise of system administration.
SuMegha Lab: Cloud Lab Kit
1. SuMegha Lab- Basic
– 1 node+ Storage
– 2 node+ Storage
– 3 node + Storage
2. SuMegha Lab- Advance
– VC with 1-16 node + Cloud Storage
– MPI Applications Framework
– Hadoop Applications Framework
3. Cloud Storage
4. SuMeghaLab- PSEs
– Cloud based PSE for SFM
– Cloud based PSE for NGS
5. C-DAC Cloud Servises:
– Free Softwares
– Installations & Trainings
– Support & Advise
19 Oct 2013 48 Trends Cloud Comp- NKN WS 2013
MEGHA: A Cloud interface for GARUDA
• Provides virtual Infrastructure on Garuda resource
– Supports IaaS and SaaS – Supports various OS flavors RHEL, Ubuntu and Fedora.
• Provides Virtual Clusters Inbuilt with – clustering tools – Parallel paradigm like Mpi & OpenMP. – Torque as scheduler
• Provides on demand Scientific Application like – SFM : Seasonal Forecasting Model, – FastA : Protein Sequence Analysis – Scilab : Open Source Equivalent of Matlab – VRGeo : Collaborative mapping platform for Geospatial information
• Supports OpenId for user login • Present focus includes
– Providing Scientific tools in the area of CFD, Bioinformatics, Molecular Biology, Statistical Computing etc
– VO enabled On demand Grid
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 49
eSanjeevani 3.1: Telemedicine App. on Cloud
eSanjeevani is a web based Telemedicine Application By CDAC Mohali. Hosted on: www. esanjeevani.in.
The key features include: • MIS based application • Doctor to Doctor consultation • Patient to Doctor consultation • Comprehensive EMR • Reports and Analytical tools • Appointment Scheduler • Medical Equipment Interfacing
Trends Cloud Comp- NKN WS 2013 19 Oct 2013 50
Megh Shushrut: e-Shushrut On Cloud
• e-Sushrut: C-DAC’s Hospital Management Information System. e-Shushrut hosted on cloud is named as Megh-Shushrut.
• The Megh Shushrut cater the needs of the state for State of Rajasthan, Government of National Capital Territory Delhi and North Eastern States, in the very near future
• Megh-Shushrut (MS) will be launched using Software as a Service (SaaS) model where different modules like Patient Registration, Out Patient Management, In Patient Management, Patient Billing, Laboratory Information System and EMR are deployed as service
• Megh-Shushrut will be deployed at Centralized Data Center Provided by the State
• MS has been configured such that, each hospital can have access to their data. In that way patient EHR is being maintained at the centralized data center. The implementation in these cases is supposed to state wide.
• The realization is will be very costly to build separate hardware and software infrastructure in all the hospitals across the state
• Cloud based solutions offer a Cost Effective solution
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 51
MEGHA: Cloud interface for GARUDA • Provides virtual Infrastructure on Garuda resource
– Supports IaaS and SaaS – Supports various OS flavors RHEL, Ubuntu and Fedora.
• Provides Virtual Clusters Inbuilt with – clustering tools – Parallel paradigm like Mpi & OpenMP. – Torque as scheduler
• Provides on Demand Scientific Applications: – SFM : Seasonal Forecasting Model, – FastA: Protein Sequence Analysis – Scilab : Open Source Equivalent of Matlab – VRGeo : Collaborative mapping platform for Geospatial information
• Supports OpenId for user login • Present focus includes
– Providing Scientific tools in the area of CFD, Bioinformatics, Molecular Biology, Statistical Computing etc
– VO enabled On demand Grid
19 Oct 2013 52 Trends Cloud Comp- NKN WS 2013
Governments from around the world are
starting to take notice of the flexibility,
operational benefits and substantial cost
savings that cloud computing can provide.
Trends Cloud Comp- ACC 2013
Governament Clouds
Indian Government: Heading To Cloud
• DietY declared E-governance on national cloud with all data centers connected.
• In National Network Cloud all states will have their private cloud
• National Cloud Benefits
– With Reduced IT management load agencies can concentrate on core activities
– Elasticity
– Agility
19 Sept 2013 Trends Cloud Comp- ACC 2013 54
Govt. Applications which can benefit on CLOUD
http://www.cisco.com/web/strategy/docs/c11-687784_cloud_omputing_wp.pdf
• Pension
• Taxes
• Education Portal with e-learning virtual class room
• Health domain – connect remote medical institutes, real time patients data analysis and treatment, medicine –at-a-distance
• Databases on Cloud- Can Host the Citizens details Will ease the process of sharing data between different public sector organisations e.g. pension, elections , Driver’s License
• etc .
http://www.computerweekly.com/blogs/public-sector/2011/02/g-cloud-introducing-the-neo-
da.html
19 Sept 2013 Trends Cloud Comp- ACC 2013 55
Government Cloud Services
Government Cloud Computing Services Can Be Categorized as:
Strategic Services
Which form only 10%
Financial Initiatives (Can be planned Over Private Cloud Networks)
Public Services
Income Tax Returns Filing
National Level Examination Conduction
Post Office Services
Health Care Services
IT services- S/W testing Utilities (Sporadic Nature of Usage)
UIDAI INITIATIVE: Aadhaar is a 12-digit
unique number which the Unique Identification Authority of India (UIDAI) will issue for all residents. The number will be stored in a centralized database and linked to the basic demographics and biometric information – photograph, ten fingerprints and iris – of each individual.
NKN: National Knowledge Network: NKN will interconnect all the research, higher education and scientific institutions in the country, over a period of three years. Cloud Computing and NKN: We are learning creation, operation and negotiations of Virtual organizations (VO) in NKN
-P.S.Dhekne,Raja Ramanna Fellow, BARC
Ref. No 4
Trends Cloud Comp- ACC
2013
• In internet Computers are attached • Internet of things, things around us
like car, refrigerator etc will be attached.
• IoT Consists any device with the ability, to gather and process information, and communicate it across the network.
58 19 Oct 2013 Trends Cloud Comp- NKN WS 2013
How Does This Work? • Different types of Devices
comprise a network (LAN, ZigBee as shown in Left)
• Applications may have different interfaces: PC, cellphone, tablets etc
• Devise networks communicate with the Applications via Internet
59 19 Oct 2013 Trends Cloud Comp- NKN WS 2013
Problem With this Approach?
To connect to Internet and Communicate Information to Applications, we need back-haul network (Circle in figure).
The many types and individual components of infrastructure can create massive complexities for solution providers.
Challenges: Scalability, less usability of resources, Security, Interoperability Solution ? An Infrastructure that can address above challenges ?
19 Oct 2013 60 Trends Cloud Comp- NKN WS 2013
On Demand Cloud
1. IaaS (Infrastructure as a Service)
2. User worry only about the Application & not about complex infrastructure
On Demand Cloud for Internet of Things
61 19 Oct 2013 Trends Cloud Comp- NKN WS 2013
Internet of Things on Cloud
The individual devices will gather the information and send it to the cloud, the application will read the information from the cloud and show it in different interfaces.
The features of Internet of things on Cloud are:
1. Scalability,
2. Flexibility
3. Better Resource Utilization,etc
62 19 Oct 2013 Trends Cloud Comp- NKN WS 2013
Applications of Internet Of Things
Smart Parking:
Monitoring of parking spaces availability in the city
Traffic Congestion:
Monitoring of vehicles and pedestrian levels to optimize driving and walking routes.
Radiation Levels
Distributed measurement of radiation levels in nuclear power stations surroundings to generate leakage alerts.
Fall Detection(e-Health)
Assistance for elderly or disabled people living independent.
63 19 Oct 2013 Trends Cloud Comp- NKN WS 2013
Sensor Clouds • Sensor Clouds: A new paradigmfor cloud computing, using sensors to collect
information of physical things and putting all sensor data into cloud computing platforms
• Efficiently handle sensor data for monitoring and mining in clouds – Efficient storage and retrievals – Partition schemes for MapReduce – Deal with real-time sensor data
65
Sensor networks
http://research.microsoft.com/en-us/events/cloudfutures2010/peng.pdf
19 Oct 2013 Trends Cloud Comp- NKN WS 2013
Sensor Cloud Infrastructure http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5635688&tag=1
• Sensor-Cloud infrastructure (SCI): manages physical sensors on IT infrastructure.
• Sensor-Cloud Infrastructure virtualizes a physical sensor as a virtual sensor on the cloud computing.
• Dynamic grouped virtual sensors on cloud computing can be automatic provisioned when the users need them.
• The approach enables the sensor management capability on cloud computing.
• Since the resource and capability of physical sensor devices is limited,
• the cloud computing on the IT infrastructure can be behalf of the sensor management such as availability and performance of physical sensors.
Fig.Sensor Cloud Infrastructure
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5635688&tag=1
19 Oct 2013 66 Trends Cloud Comp- NKN WS 2013
Sensor Clouds • The number of sensors is increasing for various purposes. • Sensors sensing the phenomenon are used by a lot of services such as
environment, healthcare and government services. • Because each sensor is closely linked to each service directly, other services
cannot use sensors easily. • Sharing sensors by other unlimited services will accelerate service innovation. • SCI is the extended cloud computing to manage sensors. • SCI provides sensor devices as a part of IT resources (CPU, memory and disk)
for end users. • SCI enables to provision service instances automatically, to monitor sensors
and to control sensors. • These functions can be used via the user interface via web browser. • IoT envision that sensors to be attached everywhere. In such environment
owner will be able to generate data. • Sensing as Service Model Evolving: can be done at personal , private and
public organization level. • Owner of sensors will be able to generate data and get a return on
investment.
67 19 Oct 2013 Trends Cloud Comp- NKN WS 2013
Sensing As A Service Model
68
• Service instances can be
created automatically when
they are requested.
• Automation is an important
characteristic of CC service
CCS.
• The service delivery time and
the service cost are improved
by automation because of no
human operations.
• Standardization is important
characteristic of CC service.
• Service providers provide
services with different
configurations by each
service requester’s
requirements before CCS
19 Oct 2013 Trends Cloud Comp- NKN WS 2013
Sensing As A Service Model
69
• Service instances can be
created automatically when
they are requested.
• Automation is an important
characteristic of CC service
CCS.
• The service delivery time and
the service cost are improved
by automation because of no
human operations.
• Standardization is important
characteristic of CC service.
• Service providers provide
services with different
configurations by each
service requester’s
requirements before CCS
19 Oct 2013 Trends Cloud Comp- NKN WS 2013
Fujitsu Laboratories: Sensor Cloud
70
• Its R&D Provide users with
information, in most appropriate
/desirable forms in accordance
with time/place/occasion, based
on:
• analysis of cloud-
accumulated sensing data
• accumulated via networks,
• by sensing individuals'
activities
• and user environments
• through sensors in mobile
devices and placed at
various sites
• Human-friendly services are in demand for fields such as healthcare, transport,
agriculture, and electrical utilities, in which ICT systems had not been fully
leveraged.
• Fujitsu Laboratories employs sensing technologies for R&D to enable
comprehension and awareness of activities, behavior, and intent of individuals
active in the real world.
M2M Sensors into Cloud M2M Smart Services Developer Kit fully supports cloud-based Cumulocity M2M Application
Platform, which delivers server-side database platform, network agents, and sensor libraries
to bring M2M connected sensors and devices into cloud. Server-side database platform
collects and analyzes data from Developer Kit connected sensors and devices and controls
devices in secure and scalable manner so that service kit is tailored for vertical M2M
industries such as fleet management and smart energy grids. 19 Oct 2013 Trends Cloud Comp- NKN WS 2013
Other Sensor Services in Cloud
71
IBM Management System Infrastructure as a Service for IT & Sensors
• The purpose is to enable the enhanced CC system that manages sensor devices
as a part of IT resource in the same way as CPU, storages, and networks to
provide the sensor devices as an infrastructure for users.
• This Infrastructure software includes three key features as cloud computing;
1. Virtualization: Multiple users can share the sensor devices.
2. Automation: The system configures the sensor devices and software
instead of the users.
3. Standardization: VM template has not only software and also the sensor
devices. Users can utilize the sensor devices easily and rapidly in this
cloud computing environment as well as IT resources
M2M Sensors into cloud: M2M Smart Services Developer Kit fully supports
• Cloud-based Cumulocity M2M Application Platform.
• Delivers server-side database platform, network agents, and sensor libraries to
bring M2M connected sensors and devices into cloud.
• Server-side database platform collects and analyzes data from Developer Kit
connected sensors and devices and controls devices in secure and scalable
manner so that service kit is tailored for vertical M2M industries such as fleet
management and smart energy grids.
19 Oct 2013 Trends Cloud Comp- NKN WS 2013
Sensing As A Service & BigData
• Modern world is full of devices comprising sensors, data
processor.
• Such resources enable sensing, capturing, collection, and
processing of real time data from billions of connected
devices.
Fact:
• 2010-Total data on earth exceeded one ZB (zettabyte).
• End of 2011- Total data number grew up to 1.8 ZB.
• 2020- Expected that the number will reach 35 ZB in 2020
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 72
Sky Computing: Distributed Cloud Computing
HTC Scientific Computng in a Distributed Cloud Environment General supporting wide range of applications
19 Oct 2013 73 Trends Cloud Comp- NKN WS 2013
Conclusion
• Next Generation Complex Systems are becoming a Reality – Peta/ExaScale Computing Systems – Combines Big Compute & Big Data to deliver – Exascale Performance in Supercomputers
• National Missions are Emerging
– National Grid Computing:GARUDA – National Knowledge NetWork:NKN – National Supercompution Mission – National BioDiversit Organization – National Cloud Innovation Council – National BigData Mission ?
• HPC Grid and Cloud infrastructures are converging:
– Compute Resources on demand to Grids – Cloud Bursting Research: Grids Vs Clouds – On Demand Cloud Services for IoT , Cloud Services for BigData
• Newer Services Models “Sensing as a Service” are Evolving • New Science “Big Science” Evolving • C-DAC is Engaged in: HPC, Grid, Cloud, BigData, and IoT
19 Oct 2013 Trends Cloud Comp- NKN WS 2013 76
C-DAC Dissemination Activities
29 April 2013 NCIC CDAC Cloud Initiatives 77
Think Parallel Parallel Programming for Engineers and Scientists
• Two Week training for Engineering College faculty, Scientific Community from R&D Labs.
• 3 Times Conducted, 180+ Teachers & Scientists are Trained on Parallel Programming Theory & Hands-on: o Parallel Programming Models o Parlllel Computer Architectures o Hybrid Systems o Cluster, Grid and Cloud Computing
• Recently conducted - June10- 20th 2013
PARCOMPTECH • Two-days Conference on Parallel Computing Technologies • Objective – to encourage National level research • Four tutorials
– Think Parallel, Big Data, TI Multicore, Cloud Computing
78
Think Parallel Think Parallel: Parallel Programming for Engineers and Scientists
(10June 2013 -20 June2013)
• Advanced Computer Architectures
• Introduction to Parallel Programming
• Principles of Parallel Algorithm Design
• Building Compute Clusters
• Numerical Computing
• Programming Paradigms
OpenMP
Message Passing Interface (MPI)
Advanced MPI
• Performance Analysis and
• Debugging Parallel Programs
• Parallel Applications/Case Studies
• GPGPU & CUDA programming
• Grid Computing
• Cloud Computing
29 April 2013 NCIC CDAC Cloud Initiatives
29 April 2013 NCIC CDAC Cloud Initiatives 79
C-DAC, a Scientific Society of Ministry of Communications and Information Technology, an organization known for its expertise in High
Performance Computing, is organizing the 2nd National Conference on “Parallel Computing Technologies (PARCOMPTECH-2014) during
27th & 28th Feb 2014 at Pune, India.
The conference aims to provide a platform for Indian scientific and engineering computing fraternities from academics, scientific labs, R&D institutions, industry and government to share their research work, knowledge and experience in the field of parallel computing technologies. This conference will act as a ground to create a collaborative environment, ultimately promoting scientific and technological development. Authors are invited to submit original, unpublished research manuscripts demonstrating their current research in parallel computing technological area.
The conference topics of interest include but not limited to:
oParallel and Distributed Architectures oMulti-core computing oGiant scale computing oInterconnection networks oParallel I/O and Storages Systems, Databases oGPU / Accelerators/ Heterogeneous computing oParallel software – Algorithms, Compilers, OS, Programming languages and libraries oGrid and Cloud computing oMiddleware for Cloud and Grid oPower optimization and Green computing oSelf aware and Self healing systems oFault-Tolerant, Scheduling and Load balancing Algorithms oPerformance Analysis, Benchmarking. oPeta and Exascale Computing oScientific/Engineering/Commercial Applications and Workflows
For accepted papers, at least one of the authors need to register and present their work at the conference. Selected papers will be published through IEEE Xplore and PARCOMPTECH Conference Proceedings. Further instructions for PARCOMPTECH2014 paper
submission is available at the following web link :http://parcomptech.cdac.in/
Conference Dates: 27th & 28th February 2104 Venue: YASHADA, Baner Road, Pune, India.
• GARUDA Challenge
• GARUDA NKN Meet – 75 R&D labs and academic institutions
– 1000+ users
– Application Drivers • OSDD • Bioinformatics • CFD • Weather • Material Science
• GARUDA Boot Camps
• CDAC-Industry Collaboration Workshops – Texas Instruments(Multicore DSP), Intel (Smart Grid), ARM, NVIDIA,
etc.
GARUDA Dissemination Works
29 April 2013 NCIC CDAC Cloud Initiatives 80
Dissemination/ Conference/Workshops/Tutorials
• Prahlada Rao B.B., “HPC for BigData” International Workshop on Data Analytics and Applications (IWDAA) 2013, BITS, Goa Campus, 1st Mar2013.
• Prahlada Rao B.B., Payal Saluja, Vineeth Simon Arackle, Tutorial “Cloud Computing for HPC” C-DAC’s National Conference on Parallel Computing Technologies (ParCompTech2013), NIAS, Bangalore, 21-23rd Feb 2013.
• Prahlada Rao B.B, Payal Saluja, Neetu Sharma, Ankit Mittal, Shivay Veer Sharma, “Cloud Computing for IOT & Sensing Based Applicationss”, in IEEE International Conference on Sensing Technologies, ICST 2012, Kollkata, India, Dec 2012.
• Prahlada Rao B.B., Payal Saluja, Vineeth Simon Arackle, Half-Day Tutorial “Open Source Platforms for Scientific Clouds” CSI’s Annual Conference on Advances in Cloud Computing (ACC2012), 26th July 2012, Bangalore.(60+ participants from Industry, Academia)
• Payal Saluja, Prahlada Rao, Ankit Mittal, Rameez Ahmad, “CDAC Scientific Cloud: On Demand Provisioning of Resources for Scientific Applications”, in Proceedings of 18th International Conference PDPTA 2012, Las vegas, Nevada, USA, 16-19 July 2012.
• Prahlada Rao B.B., Tutorial On “Cloud Computing”, International Conference on Trends in Knowledge and Information Dynamics, ICTK 2012, organized by ISI, NIMHANS Conv Centre, Bangalore, India, 10 July 2012.(150 participants from Industry, Academia)
• Dr. Prahlada Rao BB, Cloud Computing for Telemedicine, National Workshop on Telemedicine Today & Tomorrow, C-DAC Mohali, An Invited Talk, 6th July 2012.
• Prahlada Rao B. B. , National Workshop on Cloud Computing, conducted at CDAC Knowledge Park, Bangalore, 21st May 2011 .(80+ participants from Industry, Academia)
• Vineeth Simon Arackal, Arunachalam B, Payal Saluja, Prahlada Rao B B, Co-existence of Cloud and Grid: A Case Study in Service Oriented Grid GARUDA. The International Conference on Information Technology- New Generations (ITNG 2011), Las Vegas, USA. 11-13April 2011.
• Vineeth Simon Arackel, “ Coexistence of Grid and Cloud” at "Grid To Cloud (G2C)- Challenges and Applications" Workshop at VIT, Vellore, 2nd April 2011.
29 April 2013 NCIC CDAC Cloud Initiatives 81
19 Oct 2013 Trends Cloud Comp- NKN WS 2013
Thank You
82
Dr. B.B Prahlada Rao C-DAC Bangalore
email: [email protected]