Social Cloud: Facilitating “Trustworthy” Compute

& Data Resource Sharing

Presented by Daniel S. Katz*

Most work performed by:Kyle Chard, Simon Caton, Omer Rana, Kris Bubendorfer,

Ioan Petri, Ivan Rodero, Magdalena Punceva, Manish Parasharhttp://www.facebook.com/SocialCloudComputing

4th GEOSS Science and Technology Stakeholder Workshop

*Work by Katz was supported by the National Science Foundation while working at the Foundation. Any opinion, finding, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

How does resource sharing happen today

Required:

Available:

Where to get these resources?

Buy servers

Grid

Cloud

Volunteer1,2

Scenario:A user wants to backup data, but doesn’t have the required amount of computational resources/storage capacity

1: Anderson 2002; 2: Stainforth 2002

Existing approaches all have certain shortcomings

Use the (spare) capacity of friends and extend the Volunteer Computing approach to bilateral and multilateral exchanges?

Buy servers

Grid

Cloud

Volunteer

Data control

Willingness to provide resources

Access to (heterogeneous) resources

Resources available on demand

Costs, inflexibility

Exchange not necessarily bilateral

QoS, user-friendliness, access

Trust assumptions

Many years of Cloud computing … and the same old hurdles

• Security (still a major limitation)• Lack of Customisability• Economics

– Small scale consumers have requirements that may not match providers offerings

– Providers have explicit incentives to lock in consumers

– Still no true open cloud market

• Trust– Always assumed at some level– Can have anonymity

(Market-based/broker allocation)• But has new levels of trust needed

– Many models fall apart when trust is not assumed

The vision of a social cloud

• Definition: (based on Chard et al. 2011)“Social Clouds are a scalable, dynamic and user-centric resource sharing framework in which computational resources, services and information are shared amongst members on the premise of the relationships encoded in a social network.”

Socially-oriented Sharing Platform

Social Clouds enable the sharing of (heterogeneous) resources in a framework where the social structures infer an implicit level of trust

Social Cloud – building on existing Social Network Platforms

Users contribute to “good” causes

Resources are idle 40-95%1,000,000,000

Users

On average 190 friends

Ubiquitous: Facebook ~ 1.3B users (Q2, 2014)Represent mostly pre-existing real world relationshipsHave notions of pre-existent trust fabric inherently interwoven into the network structureMany applications now use social networks as a platform for:

Authentication e.g. Facebook ConnectOnline Presence e.g. fb.com/your_page, Google Places (API)Application Portals e.g. progress thru processors, ASPEN and PolarGrid project

Social Clouds ... Via Edge Devices, not Data Centers

• Cloud computing is built around large datacenters– Distributed, but still built with

capacity limits

• Expanding the boundaries of clouds across edge devices

• Devices can vary in scope:– Set-top boxes(from 500 GB to 1 TB of storage)– Media Centres (Home PCs)– Can use content distribution

networks (Netflix, HTTP, YouTube)

21.04.23 7

Social Clouds: two aspects/research challenges

• Resource management– Edge Devices – with Peer-

2-Peer-like interaction– Means of content

distribution– Provides basis for a

“credits” and revenue model

– Could be via social group or facilitated by content provider (e.g. Virgin Media)

• Access– Use of Social Media (e.g.

Facebook) app. to utilise friends network

– Use trust relationships within an existing social network to discover partners for interaction

– Use of a “virtual currency” (credit tokens) that facilitates trading within a local community

Social Clouds: types of data(e.g. podcasts, blogs, photos, music, documents)

• User generated content may be:– Intermittent - often required at different time

intervals but not continuously– Temporary - required for a short period of time

(e.g. processing memory for running an experiment) and often only once

– Backup - required with the highest security implications and privacy

– Working - can be accessed in real time and continuously

Incentives

Family, close friends

Friends, Colleagues

Other People

Different groups have different levels of perceived trustChoice of interaction mechanism could depend on relationship type

Example:•Non-monetary incentives for close friends•Monetary incentives for other people

Volunteer

Trophy

Posted Price

Reciprocal

Auction

Relationship to GEOSS(http://www.earthobservations.org/geoss.php)

• Variety of data sets– at different locations (“a network of content providers”)

• Linking “system of systems”• Enable multiple types of specialist, application specific,

decision support – Community provision of algorithms + “shim” (translation

services)

• Trust issues seem to be key– Who: (i) generates, (ii) hosts; (iii) caches the data set– Provenance issues associated with data– Hosted on devices with varying capability (and reliability/

availability profiles)Similar requirements to Social Clouds

Investigations/Talk Structure

• Social Cloud platform– Data Storage: Facebook app.– Computational: Seattle-based VMs – Use of Social Clouds to support Content Distribution (user-

side version of the “Cloud Files” implementation)• Resource trading scenarios

– Eigentrust (direct interaction + recommendations and feedback)

– Complementary (“virtual”) currency• Trust relationships (incentive models)

– Game theoretic models (incentives to provide incorrect feedback) + number of malicious peers

The Platform

Data Storage: Facebook app.

Platform Architecture

Social MarketplaceMatching supply with demandProtocols for resource allocation, rules of exchange, information store/registry

Platform ManagerOverall system coordination

Socio-Technical AdapterIdentity verification (e.g. OAuth)Connect to various “types” of social networks (DBLP, Facebook, Foursquare, etc)

Resource FabricsVirtualisation & Sandboxing mechanismIntegration with Globus end points, Seattle VM, etc

Social Cloud: Platform for Data Storage (Demonstrator)

• Simple Storage Service Implemented as a Facebook application• Use Case: a back up facility

Agreement

Kyle Chard, Kris Bubendorfer, Simon Caton, Omer F. Rana, “Social Cloud Computing: A Vision for Socially Motivated Resource Sharing”. IEEE Transactions on Services Computing 5(4): 551-563 (2012)

Posted Price

Storage

Social CloudSocial Cloud

Storage

Storage

MDSMDS

User ID URL Capacity Price

User1 100 MB 5

User2 500 MB 10

User3 5 GB 7

– Enables interactions based upon active trading/collaborative decisions

– Intuitively facilitates reciprocal collaboration– Current “norm” in industry solutions

Dynamic Auctions

• Auction:– Enables dynamic participant pairing– Sealed bid second price reverse auction

• Could be extended to any other auction mechanism

Application

Posted Price Scalability

• Varying the size of the MDS and number of matches• With a size of 2000, 100 matches can be discovered

in ~ 2 seconds

The Platform

Computational: Seattle-based VMs

Processor Sharing – via extended Seattle VMs

• Matching between users & owners• Seattle (https://seattle.poly.edu/html/ ) – Open P2P platform

– Seattle “Clearing house” mechanism. 10 “vessels” (VMs) for each new install– Node Manager: gatekeeper for resources deployed on every contributed resource (credential checking for VM

interaction)– Host machine location (in a lookup service) + Public/Private keys generated– Repy (Reduced Python for sandboxed environments)

Processor Sharing – via extended Seattle VMs

• Identify list of donation nodes• Filter list based on “friends list” for a particular user • Match mechanism

– Select consumer preferences for each friend– Select preferences for each friend for requesting user

• Extends Seattle’s implementation of (pseudo) random allocation to reduce user/donation permutations

Simon Caton, Christian Haas, Kyle Chard, Kris Bubendorfer, Omer F. Rana, “A Social Compute Cloud: Allocating and Sharing Infrastructure Resources via Social Networks”. IEEE Transactions on Services Computing 7(3): 359-372 (2014)

The Platform

Use of Social Clouds to support Content Distribution (user-side version of the “Cloud Files” implementation)

Social CDN – Use Case

• Enable content distribution through user-supplied storage– Home users (DSL-based)– National Labs (dedicated networks) – often related to a

particular project (e.g. D0)• Variable availability profiles

– Bandwidth throttling with DSL-based set up– Establish a VPN between contributing sites (use of

SocialVPN)• Identify:

– Number of replicas needed to ensure availability• Served-based approach

– “CloudFiles” in RackSpace (OpenStack) – use of Akamai CDN

A Social Content Delivery Network for Scientific Cooperation

Replica Placement:•Random•Node Degree: highest no. of edges•Community Node Degree (highest degree within a community, i.e. no adjacent placement)•Clustering Coefficient (similar to highest betweenness scores)

Scenario and Community Representation

26

• Baseline Graph: DBLP publications graph (Kyle): 3 degrees (2009-10)– Nodes: authors, Edges: coauthorship of 1 or more papers

• Double co-authorship: at least 2 publications• No. of Authors: < 6 authors on the paper• Trust: captured through prior collaborative work• Having constructed a network, we assign replicas, and then test with publications

from 2011

Kai Kugler, Simon Caton, Kyle Chard, Daniel S. Katz, "On Replica Placement in a Social CDN for e-Science,” 10th IEEE International Conference on eScience, 2014.

Double Coauthorship No. of Coauthors

Results (at least 60 repetitions)

27

Kyle Chard, Simon Caton, Omer Rana and Daniel S. Katz, “A Social Content Delivery Network for Scientific Cooperation: Vision, Design, and Architecture”, 3rd Int. Workshop on Data Intensive Computation in the Cloud (DataCloud), in conjunction with ACM/IEEE SC12 conference, Salt Lake City, November 2012

Data “Followers” & “Survivability” dynamics(The “Data Wildfire”)

• Register interest in a data set– Equivalent to “Like” (Facebook) and “Favorite”

(Twitter) – Event generated on subsequent update on a data set

• Enable “interesting” data set to be propagated– Equivalent to a “Share” (Facebook) and “Retweet”

(Twitter)– Enables data sets with community interest to become

popular over time• Can be useful as a basis to support resource allocation

Finding Brokers

Broker Emergence

• Finding suitable providers– Centralized (registry/index)– Distributed (flooding, gossip protocol, federated registry

services)• Brokers can utilise both centralized and distributed

capability– Brokers influence interaction dynamics in the network– Predefine broker nodes at start up

• How can trading within a Social Cloud be enhanced by dynamic emergence of “brokers”– Which nodes could be more useful Brokers?

Take away message

• Dynamic selection of brokers based on their position in the network– Incentivise nodes to become broker based on increase in revenue– Role adaptation in the network – buyers/seller broker– Use of “influence” metrics based on a social score

• Dynamic network properties may lead to limited benefit with pre-defined brokers

• the social score algorithm generates a higher volume of trades than the dominating set algorithm.

• The performance differences of Social Score vs Dominating set are determined by two aspects:– Graph properties and– Choice of evaluation metrics.

Ioan Petri, Magdalena Punceva, Omer F. Rana, George Theodorakopoulos: Broker Emergence in Social Clouds. IEEE CLOUD 2013, San Jose, CA, USA, pp 669-676

Incentive Models

Incentives & Trading

In Conclusion

• Social Clouds provide an important user-driven alternative to data-center based Clouds– E.g., Wuala networks (NL), AmazingStore (China), etc

• Issues of Trust, Reputation and Economic incentives are key– Include other factors: availability, reliability, uptime, power

usage, etc.– Softer than traditional Service Level Agreement model

• Current focus: Broker “emergence” in Social Clouds– Identify dominating sets in a social graph– Implementation using CometCloud

Further Reading

D. Neumann, C. Bodenstein, O. F. Rana, R. Krishnaswamy, ”STACEE: Enhancing Storage Clouds using Edge Devices”.IEEE/ACM Workshop on Autonomic Computing for Economics (ACE 2011) alongside ICAC 2011, Karlsruhe, Germany, 14 June 2011. ACM Press.

Kyle Chard, Kris Bubendorfer, Simon Caton, Omer Rana, “Social Cloud Computing: A Vision for Socially Motivated Resource Sharing,” IEEE Transactions on Services Computing, 2011.

Ioan Petri, Omer Rana, Gheorghe Cosmin Silaghi: “SLA as a Complementary Currency in Peer-2-Peer Markets”, Proceedings of GECON 2010: 141-152, Springer Verlag.

“Trust Modelling and Analysis in Peer-to-Peer Clouds” Ioan Petri, Omer Rana, Yacine Rezgui, and Gheorghe Cosmin Silaghi, Journal of Cloud Computing, Inderscience, 2012

Omer Rana and Simon Caton, ”Business Models for On-line Social Networks: Challenges and Opportunities”, International Journal of Virtual Communities and Social Networking, July-September 2010, Vol.2, No.3, pp 31-42

Ioan Petri, Omer F. Rana, Gheorghe Cosmin Silaghi: Service level agreement as a complementary currency in peer-to-peer markets. Future Generation Comp. Syst. 28(8): 1316-1327 (2012)