cloud computing - summary · 2016-03-29 · internet of things (iot) • “the internet of things...
TRANSCRIPT
Basics of Cloud Computing – Lecture 8
Mobile Cloud, Internet of Things and
Cloud Computing – Summary and
Preparation for Examination
Satish Srirama
Outline
• Continue with Research at Mobile & Cloud Lab
• Quick recap of what we have learnt as part of
this course
• How to prepare for the examination
3/29/2016 Satish Srirama 2/38
3/29/2016
[Tomi T Ahonen]
Satish Srirama 3/38
Mobile Applications
• One can do interesting things on mobiles directly
– Today’s mobiles are far more capable
– Location-based services (LBSs), mobile social networking, mobile commerce, context-aware services etc.
• It is also possible to make the mobile a service provider
– Mobile web service provisioning [Srirama et al, ICIW 2006; Srirama
and Paniagua, MS 2013]
– Challenges in security, scalability, discovery and middleware are studied [Srirama, PhD 2008]
– Mobile Social Network in Proximity [Chang et al, ICSOC 2012; PMC
2014]
3/29/2016 Satish Srirama 4/38
However, we still have not achieved
• Longer battery life
– Battery lasts only for 1-2 hours for continuous computing
• Same quality of experience as on desktops
– Weaker CPU and memory
– Storage capacity
• Still it is a good idea to take the support of external resources for building resource intensive mobile applications
3/29/2016 Satish Srirama 5/38
Mobile Cloud Applications
• Bring the cloud infrastructure to the proximity
of the mobile user
• Mobile has significant advantage by going
cloud-aware
– Increased data storage capacity
– Availability of unlimited processing power
– PC-like functionality for mobile applications
– Extended battery life (energy efficiency)
3/29/2016 Satish Srirama 6/38
Mobile Cloud – Our interpretation
• We do not see Mobile Cloud to be just a scenario where mobile is taking the help of a much powerful machine!!!
• We do not see cloud as just a pool of virtual machines
• Mobile Cloud based system should take advantage of some of the key intrinsic characteristics of cloud efficiently
– Elasticity & AutoScaling
– Utility computing models
– Parallelization (e.g., using MapReduce)
3/29/2016 Satish Srirama 7/38
Mobile Cloud Binding Models
3/29/2016
Task Delegation Code Offloading
[Flores & Srirama, JSS 2014]
[Flores et al, MoMM 2011]
Mobile Cloud
[Flores and Srirama, MCS 2013]
Satish Srirama 8/38
MCM – enables
• Interoperability between different Cloud Services (IaaS, SaaS, PaaS) and Providers (Amazon, Eucalyptus, etc)
• Provides an abstraction layer on top of API
• Composition of different Cloud Services
• Asynchronous communication between the device and MCM [Warren et al, IEEE PC 2014]
• Means to parallelize the tasks and take advantage of Cloud’s intrinsic characteristics
3/29/2016
[Flores et al, MoMM 2011]
Satish Srirama 9/38
MCM applications
• CroudSTag [Srirama et al, MobiWIS 2011]
– Social group formation with people identified in Pictures/Videos
• Zompopo [Srirama et al, NGMAST 2011]
– Intelligent calendar, by mining accelerometer sensor data
• Bakabs [Paniagua et al, iiWAS-2011]
– Managing the Cloud resources from mobile
• Sensor data analysis– Human activity recognition
– Context aware gaming
– MapReduce based sensor data analysis [Paniagua et al, MobiWIS 2012]
• SPiCa: A Social Private Cloud Computing Application Framework [Chang et al, MUM 2014]
3/29/2016 Satish Srirama 10/38
Code Offloading - Major Components
• Major research challenges – What, when,
where and how to offload?
• Mobile– Code profiler
– System profilers
– Decision engine
• Cloud based surrogate platform
3/29/2016
[Flores and Srirama, MCS 2013]
Satish Srirama 11/38
Challenges and technical problems
• Inaccurate code profiling– Code has non-deterministic behaviour during runtime
• Based on factors such as input, type of device, execution environment, CPU, memory etc.
– Some code cannot be profiled (e.g. REST)
• Integration complexity– Dynamic behaviour vs Static annotations
• E.g. Static annotations cause unnecessary offloading
• Dynamic configuration of the system
• Offloading scalability and offloading as a service– Surrogate should have similar execution environment
– Should also consider about resource availability of Cloud
3/29/2016
[Flores et al, IEEE Communications Mag 2015]
Satish Srirama 12/38
Practical adaptability of offloading
Applications that can benefit became limited with increase in device capacities3/29/2016 Satish Srirama 13/38
Internet of Things (IoT)
• “The Internet of Things allows people and
things to be connected Anytime, Anyplace,
with Anything and Anyone, ideally using Any
path/network and Any service.” [European
Research Cluster on IoT]
• More connected devices than people
• Cisco believes the market size will be $19
trillion by 2025
3/29/2016 Satish Srirama14/38
IoT - Scenarios
• Environment Protection
• Smart Home
• Smart Healthcare
• Smart Agriculture
[Kip Compton][Perera et al, TETT 2014]
3/29/2016 Satish Srirama15/38
Internet of Things – Challenges
Sensors Tags Mobile Things
Appliances & Facilities
How to interact
with ‘things’
directly?
How to provide
energy efficient
services?
How do we
communicate
automatically?
[Chang et al, ICWS 2015]
[Chang et al, SCC 2015;
Liyanage et al, MS 2015]
3/29/2016 Satish Srirama16/38
Cloud-based IoT
Sensing and smart devices
Connectivity nodes &
Embedded processing
Remote Cloud-based
processing
Proxy Storage
Processing
3/29/2016 Satish Srirama 17/38
Further research focus in IoT
• We have established IoT and Smart Solutions
Lab with Telia company support
• Interesting topics
– Discovery of IoT devices
– Study of IoT based middlewares
– Study of available IoT platforms
• Amazon IoT
• Open IoT
3/29/2016 Satish Srirama18/38
IoT Data Processing on Cloud
• Enormous amounts of unstructured data– In Zetabytes (1021 bytes) by 2020 [TelecomEngine]
– Has to be properly stored, analysed and interpreted and presented
• Big data acquisition and analytics– Is MapReduce sufficient?
• MapReduce is not good for iterative algorithms [Srirama et al, FGCS 2012]
– IoT mostly deals with streaming data• Message queues such as Apache Kafka can be used to buffer and feed
the data into stream processing systems such as Apache Storm
• Apache Spark streaming
• How to ensure QoS aspects such as security of data?– Anonymization and Expiry of data?
• Especially for the personal data
3/29/2016 Satish Srirama19/38
WHAT WE LEARNT IN THE COURSE!
3/29/2016 Satish Srirama 21
What is Cloud Computing?
• Computing as a utility
– Consumers pay based on their usage
• Cloud Computing characteristics
– Illusion of infinite resources
– No up-front cost
– Fine-grained billing (e.g. hourly)
• Gartner: “Cloud computing is a style of computing where massively scalable IT-related capabilities are provided ‘as a service’ across the Internet to multiple external customers”
3/29/2016 Satish Srirama 22/38
Cloud Computing - Services• Software as a Service – SaaS
– A way to access applications hosted on the web through your web browser
• Platform as a Service – PaaS– Provides a computing platform
and a solution stack (e.g. LAMP) as a service
• Infrastructure as a Service –IaaS– Use of commodity computers,
distributed across Internet, to perform parallel processing, distributed storage, indexing and mining of data
– Virtualization
SaaS
Facebook, Flikr, Myspace.com,
Google maps API, Gmail
PaaS
Google App Engine, Force.com, Hadoop, Azure,
Heroku, etc
IaaS
Amazon EC2, Rackspace, GoGrid, SciCloud, etc.
Level of
Abstraction
3/29/2016 Satish Srirama 23/38
Cloud Computing - Themes
• Massively scalable
• On-demand & dynamic
• Only use what you need - Elastic – No upfront commitments, use on short term basis
• Accessible via Internet, location independent
• Transparent – Complexity concealed from users, virtualized,
abstracted
• Service oriented – Easy to use Service Level Agreements
3/29/2016 Satish Srirama 24/38
Cloud Computing Progress
• Short term and long term implications of cloud
• Economics of cloud users and cloud providers
• Challenges and opportunities offered by cloud
computing
3/29/2016 Satish Srirama 25/38
Cloud Providers
• Amazon Web Services– EC2, S3, EBS, Elastic Load Balancing, Amazon Auto Scale,
Amazon CloudWatch, IAM, Elastic Beanstalk, CloudFormation, Simple Workflow Service, Elastic MapReduce
• Private Cloud enabling technologies– Eucalyptus
– OpenStack• Worked with SciCloud
• PaaS– Google AppEngine
– Windows Azure
3/29/2016 Satish Srirama 26/38
Scaling Applications on the Cloud
• Two basic models of scaling
– Vertical scaling, aka Scale-up
– Horizontal scaling, aka Scale-out
• Scaling Enterprise Applications in the Cloud
• Load balancing
– Types and algorithms
• Autoscaling
3/29/2016 Satish Srirama 27/38
Economics of Cloud Providers
• Cloud Computing providers bring a shift from high reliability/availability servers to commodity servers– At least one failure per day in large datacenter
• Why?– Significant economic incentives
• much lower per-server cost
• Caveat: User software has to adapt to failures– Very hard problem!
• Solution: Replicate data and computation– MapReduce & Distributed File System
3/29/2016 Satish Srirama 28/38
MapReduce
• Programmers specify two functions:map (k, v) → <k’, v’>*reduce (k’, v’) → <k’, v’>*– All values with the same key are reduced together
• The execution framework handles everything else…• Not quite…usually, programmers also specify:
partition (k’, number of parWWons) → parWWon for k’– Often a simple hash of the key, e.g., hash(k’) mod n– Divides up key space for parallel reduce operationscombine (k’, v’) → <k’, v’>*– Mini-reducers that run in memory after the map phase– Used as an optimization to reduce network traffic
3/29/2016 Satish Srirama 29/38
Hadoop Processing Model
• Create or allocate a cluster
• Put data onto the file system (HDFS)– Data is split into blocks
– Replicated and stored in the cluster
• Run your job– Copy Map code to the allocated nodes
• Move computation to data, not data to computation
– Gather output of Map, sort and partition on key
– Run Reduce tasks
• Results are stored in the HDFS
3/29/2016 Satish Srirama 30/38
MapReduce Examples
• Distributed Grep
• Count of URL Access Frequency
• Reverse Web-Link Graph
• Inverted Index
• Distributed Sort
3/29/2016 Satish Srirama 31/38
Synchronization in Hadoop
• Approach 1: turn synchronization into an ordering problem– Sort keys into correct order of computation
– Partition key space so that each reducer gets the appropriate set of partial results
– Hold state in reducer across multiple key-value pairs to perform computation
– Illustrated by the “pairs” approach in calculating conditional probability of words
• Approach 2: construct data structures that “bring the pieces together”– Each reducer receives all the data it needs to complete the
computation
– Illustrated by the “stripes” approach
3/29/2016 Satish Srirama 32/38
Research @ Mobile & Cloud Lab
• Scientific computing on the cloud
• Classification on how the algorithms can be
adapted to MR
• Limitations of Hadoop MapReduce
– Alternatives
• Mobile Cloud
– Mobile Cloud Binding Models
• Cloud based Internet of Things
3/29/2016 Satish Srirama 33/38
This week in lab
• Complete the work from previous labs
3/29/2016 Satish Srirama 34/38
Examination
• Exam I : Monday, 11.04.2016, 10:00 – 12:00
Room yet to be decided
• Exam II : Tuesday, 12.04.2016, 10:00 – 12:00 in
room J. Liivi 2 - 402
3/29/2016 Satish Srirama 35/38
Preparation for Examination
• One of the earlier exam paper is kept online
• Slides are mostly self sufficient
• References are mentioned for further reading
• Mainly focus at keywords
• Let us discuss some questions…
3/29/2016 Satish Srirama 36/38
References
• Basics of Cloud Computing – Lectures https://courses.cs.ut.ee/2016/cloud/spring/Main/Lectures
• List of Publications - Satish Narayana Srirama - http://math.ut.ee/~srirama/publications.html
• [Flores et al, IEEE Communications Mag 2015] H. Flores, P. Hui, S. Tarkoma, Y. Li, S. N. Srirama, R. Buyya: Mobile Code Offloading: From Concept to Practice and Beyond, IEEE Communications Magazine, ISSN: 0163-6804, 53(3):80-88, 2015. IEEE. DOI:10.1109/MCOM.2015.7060486
• [Chang et al, SCC 2015]C. Chang, S. N. Srirama, J. Mass: A Middleware for Discovering Proximity-based Service-Oriented Industrial Internet of Things, 12th IEEE International Conference on Services Computing (SCC 2015), June 27 - July 2, 2015, pp. 130-137. IEEE.
• [Liyanage et al, MS 2015] M. Liyanage, C. Chang, S. N. Srirama: Lightweight Mobile Web Service Provisioning for Sensor Mediation, 4th International Conference on Mobile Services (MS 2015), June 27 - July 2, 2015, pp. 57-64. IEEE
• [Chang et al, ICWS 2015] C. Chang, S. Loke, H. Dong, F. Salim, S. N. Srirama, M. Liyanage, S. Ling: An Energy-Efficient Inter-organizational Wireless Sensor Data Collection Framework, The IEEE 22nd International Conference on Web Services (ICWS 2015), June 27 - July 2, 2015, pp. 639-646. IEEE.
• [Flores and Srirama, JSS 2014] H. Flores, S. N. Srirama: Mobile Cloud Middleware, Journal of Systems and Software, ISSN: 0164-1212, 92(1):82-94, 2014. Elsevier. DOI: 10.1016/j.jss.2013.09.012.
• [Chang et al, PMC 2014] C. Chang, S. N. Srirama, S. Ling: Towards an Adaptive Mediation Framework for Mobile Social Network in Proximity, Pervasive and Mobile Computing Journal, MUCS Fast track, ISSN: 1574-1192, 12:179-196, 2014. Elsevier. DOI: 10.1016/j.pmcj.2013.02.004.
• [Warren et al, IEEE PC 2014] I. Warren, A. Meads, S. N. Srirama, T. Weerasinghe, C. Paniagua: Push Notification Mechanisms for Pervasive Smartphone Applications, IEEE Pervasive Computing, ISSN: 1536-1268, 13(2):61-71, 2014. IEEE. DOI:10.1109/MPRV.2014.34.
• [Chang et al, MUM 2014] C. Chang, S. N. Srirama, S. Ling: SPiCa: A Social Private Cloud Computing Application Framework, The 13th International Conference on Mobile and Ubiquitous Multimedia (MUM 2014), November 25-28, 2014, pp. 30-39. ACM.
• [Jakovits and Srirama, HPCS 2014] P. Jakovits, S. N. Srirama: Evaluating MapReduce Frameworks for Iterative Scientific Computing Applications, The 2014 (12th) International Conference on High Performance Computing & Simulation (HPCS 2014), July 21-25, 2014, pp. 226-233. IEEE.
• [Kromonov et al, HPCS 2014] I. Kromonov, P. Jakovits, S. N. Srirama: NEWT - A resilient BSP framework for iterative algorithms on Hadoop YARN, The 2014 (12th) International Conference on High Performance Computing & Simulation (HPCS 2014), July 21-25, 2014, pp. 251-259. IEEE.
• [Srirama and Viil, HPCC 2014] S. N. Srirama, J. Viil: Migrating Scientific Workflows to the Cloud: Through Graph-partitioning, Scheduling and Peer-to-Peer Data Sharing, 16th Int. Conf. on High Performance and Communications (HPCC 2014) workshops, August 20-22, 2014, pp. 1137-1144. IEEE.
• [Jakovits and Srirama, Nordicloud 2013] P. Jakovits, S. N. Srirama: Clustering on the Cloud: Reducing CLARA to MapReduce, 2nd Nordic Symposium on Cloud Computing & Internet Technologies (NordiCloud 2013), September 02-03, 2013, pp. 64-71. ACM.
• [Jakovits et al, HPCS 2013] P. Jakovits, S. N. Srirama, I. Kromonov: Viability of the Bulk Synchronous Parallel Model for Science on Cloud, The 2013 (11th) International Conference on High Performance Computing & Simulation (HPCS 2013), July 01-05, 2013, pp. 41-48. IEEE.
• [Srirama and Paniagua, MS 2013] S. N. Srirama, C. Paniagua: Mobile Web Service Provisioning and Discovery in Android Days, The 2013 IEEE International Conference on Mobile Services (MS 2013), June 27 - July 02, 2013, pp. 15-22. IEEE.
• [Flores and Srirama, MCS 2013] H. Flores, S. N. Srirama: Adaptive Code Offloading for Mobile Cloud Applications: Exploiting Fuzzy Sets and Evidence-based Learning, The Fourth ACM Workshop on Mobile Cloud Computing and Services (MCS 2013) @ MobiSys 2013, June 25-28, 2013, pp. 9-16. ACM.
• [Oliner et al, 2013] Oliner, Adam J., Anand P. Iyer, Ion Stoica, Eemil Lagerspetz, and Sasu Tarkoma. "Carat: Collaborative energy diagnosis for mobile devices." In Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems, p. 10. ACM, 2013.
• [Srirama et al, SOCA 2012] S. N. Srirama, C. Paniagua, H. Flores: Social Group Formation with Mobile Cloud Services, Service Oriented Computing and Applications Journal, ISSN: 1863-2386, 6(4):351-362, 2012. Springer. DOI: 10.1007/s11761-012-0111-5.
• [Srirama et al, FGCS 2012] S. N. Srirama, P. Jakovits, E. Vainikko: Adapting Scientific Computing Problems to Clouds using MapReduce, Future Generation Computer Systems Journal, 28(1):184-192, 2012. Elsevier press. DOI 10.1016/j.future.2011.05.025.
3/29/2016 Satish Srirama 37/38
References - continued• [Chang et al, ICSOC 2012] C. Chang, S. N. Srirama, S. Ling: An Adaptive Mediation Framework for Mobile P2P Social
Content Sharing, 10th International Conference on Service Oriented Computing (ICSOC 2012), November 12-16, 2012, pp. 374-388. Springer LNCS.
• [Paniagua et al, MobiWIS 2012] C. Paniagua, H. Flores, S. N. Srirama: Mobile Sensor Data Classification for Human Activity Recognition using MapReduce on Cloud, The 9th Int. Conf. on Mobile Web Information Systems (MobiWIS2012), August 27-29, 2012, pp. 585-592. Elsevier.
• [Srirama et al, SPJ 2011] S. N. Srirama, O. Batrashev, P. Jakovits, E. Vainikko: Scalability of Parallel Scientific Applications on the Cloud, Scientific Programming Journal, Special Issue on Science-driven Cloud Computing, 19(2-3):91-105, 2011. IOS Press. DOI 10.3233/SPR-2011-0320.
• [Flores et al, MoMM 2011] H. Flores, S. N. Srirama, C. Paniagua: A Generic Middleware Framework for Handling Process Intensive Hybrid Cloud Services from Mobiles, The 9th International Conference on Advances in Mobile Computing & Multimedia (MoMM-2011), December 5-7, 2011, pp. 87-95. ACM.
• [Paniagua et al, iiWAS 2011] C. Paniagua, S. N. Srirama, H. Flores: Bakabs: Managing Load of Cloud-based Web Applications from Mobiles, The 13th International Conference on Information Integration and Web-based Applications & Services (iiWAS-2011), December 5-7, 2011, pp. 489-495. ACM.
• [Srirama et al, MobiWIS 2011] S. N. Srirama, C. Paniagua, H. Flores: CroudSTag: Social Group Formation with Facial Recognition and Mobile Cloud Services, The 8th International Conference on Mobile Web Information Systems (MobiWIS 2011), September 19-21, 2011, v. 5 of Procedia Computer Science, pp. 633-640. Elsevier. doi: 10.1016/j.procs.2011.07.082.
• [Srirama et al, NGMAST 2011] S. N. Srirama, H. Flores, C. Paniagua: Zompopo: Mobile Calendar Prediction based on Human Activities Recognition using the Accelerometer and Cloud Services, 5th International Conference on Next Generation Mobile Applications, Services and Technologies (NGMAST 2011), September 14-16, 2011, pp. 63-69. IEEE.
• [Batrashev et al, HPCS 2011] O. Batrashev, S. N. Srirama, E. Vainikko: Benchmarking DOUG on the Cloud, The 2011 International Conference on High Performance Computing & Simulation (HPCS 2011), July 4-8, 2011, pp. 677-685. IEEE.
• [Srirama, PhD 2008] S. N. Srirama: Mobile Hosts in Enterprise Service Integration, PhD thesis, RWTH Aachen University, September, 2008.
• [Srirama et al, ICIW 2006] S. N. Srirama, M. Jarke, W. Prinz: Mobile Web Service Provisioning, Proceedings of the Advanced International Conference on Telecommunications and International Conference on Internet and Web Applications and Services (AICT-ICIW 2006), February 23-25, 2006, pp. 120-125. IEEE Computer Society Press.
• [Valiant, 1990] L. G. Valiant: A bridging model for parallel computation, Commun. ACM, vol. 33, no. 8, pp. 103–111, Aug. 1990.
3/29/2016 Satish Srirama 38/38