Data as a Service – Concepts, Design &

Implementation, and Ecosystems

Hong-Linh Truong

Distributed Systems Group,

Vienna University of Technology

truong@dsg.tuwien.ac.at http://www.infosys.tuwien.ac.at/staff/truong

1 ASE WS 2012

Advanced Services Engineering,

WS 2012

Outline

Data provisioning and data service units

Data-as-a-Service concepts

DaaS design and implementation

DaaS ecosystems

ASE WS 2012 2

Data versus data assets

ASE WS 2012 3

Data

Data Assets

Data management

and provisioning

Data concerns

Data collection,

assessment and

enrichment

Data provisioning activities and

issues

ASE WS 2012 4

Collect

• Data sources

• Ownership

• Quality assessment and enrichment

Store

• Query and backup capabilities

• Local versus cloud, distributed versus centralized storage

Access

• Interface

• Public versus private access

• Access granularity

• Pricing and licensing model

Utilize

• Alone or in combination with other data sources

• Redistribution

Non-exhausive list! Add your own issues!

Stakeholders in data provisioning

ASE WS 2012 5

Data

Data Provider

• People (individual/crowds/organization)

• Software, Things

Service Provider

• Software and people

Data Consumer

• People, Software, Things

Data Aggregator/Integrator

• Software

• People + software

Data Assessment

• Software and people

Recall – Service Unit

ASE WS 2012 6

Service model

Unit Concept

Service unit

„basic

component“/“basic

function“ modeling

and description

Consumption,

ownership,

provisioning, price, etc.

What about service units providing data?

Data service unit

ASE WS 2012 7

Service model

Unit Concept

Data service

unit

Data

Can be used for private

or public

Can be elastic or not

Data service units in clouds/internet

Provide data capabilities rather than provide

computation or software capabilities

Providing data in clouds/internet is an increasing

trend

In both business and e-science environments

Bio data, weather data, company balance

sheets, etc., via Web services

8 ASE WS 2012

Data service unit

9

Data service units in

clouds/internet

data

Internet/Cloud

Data service unit

People

data

Data service unit

Things

ASE WS 2012

data data

SO DATA SERVICE UNIT IS

BIG OR SMALL? PROVIDING

REALTIME OR STATIC DATA?

Discussion time

ASE WS 2012 10

11

NIST Cloud definitions

“This cloud model promotes availability and is

composed of five essential characteristics,

three service models, and four deployment

models.”

ASE WS 2012

Source: NIST Definition of Cloud Computing v15, http://csrc.nist.gov/groups/SNS/cloud-computing/cloud-def-v15.doc

Data as a Service -- characteristics

On-demand self-service

Capabilities to provision data at different granularities

Resource pooling

Multiple types of data, big, static or near-realtime,raw data and

high-level information

Broad network access

Can be access from anywhere

Rapid elasticity

Easy to add/remove data sources

Measured service

Measuring, monitoring and publishing data concerns and usage

ASE WS 2012 12

Built atop NIST‘s definition

Data-as-a-Service – service models

Data as a Service – service models

and deployment models

ASE WS 2012 13

Storage-as-a-Service

(Basic storage functions)

Database-as-a-Service

(Structured/non-structured

querying systems)

Data publish/subcription

middleware as a service

Sensor-as-a-Service

Private/Public/Hybrid/Community Clouds

deploy

Examples of DaaS

ASE WS 2012 14

WHAT ELSE DO YOU THINK

CAN BE INCLUDED INTO DAAS

MODELS?

Discussion time

ASE WS 2012 15

DaaS design & implementation –

APIs

Read-only DaaS versus CRUD DaaS APIs

Service APIs versus Data APIs

They are not the same wrt concerns

SOAP versus REST

ASE WS 2012 16

Example: infochimps

DaaS design & implementation –

service provider vs data provider

The DaaS provider is separated from the data

provider

17

DaaS

Consumer

DaaS

Sensor

DaaS

Consumer DaaS provider Data

provider

ASE WS 2012

Example: DaaS provider =! data

provider

18

DaaS design & implementation –

structures

DaaS and data providers have the right to

publish the data

ASE WS 2012 19

DaaS

• Service APIs

• Data APIs for the whole resource

Data Resource

• Data APIs for particular resources

• Data APIs for data items

Data Items

• Data APIs for data items

Three levels

20

DaaS design & implementation –

structures (2)

Data

items

Data

items

Data

items

Data resource

Data

assets

Data resource Data resource

Data resource Data resource

Consumer

Consumer

DaaS

ASE WS 2012

DaaS design & implementation –

patterns for „turning data to DaaS“ (1)

ASE WS 2012 21

DaaS data Build Data

Service

APIs

Deploy

Data

Service

Examples: using WSO2 data service

Storage/Database

-as-a-Service

DaaS design & implementation –

patterns for „turning data to DaaS“ (2)

ASE WS 2012 22

data

Examples: using

Amazon S3

DaaS

Storage/Databa

se/Middleware

DaaS design & implementation –

patterns for „turning data to DaaS“ (3)

ASE WS 2012 23

data

Examples: using

COSM/Pachube

Things

One thing 10000... things

DaaS

Storage/Database/

Middleware

DaaS design & implementation –

patterns for „turning data to DaaS“ (4)

ASE WS 2012 24

data

Examples: using Twitter

People DaaS

....

DaaS design & implementation –

not just „functional“ aspects (1)

ASE WS 2012 25

data DaaS .... data assets

Data

concerns

Quality of

data Ownership

Price License ....

Enrichment Cleansing

Profiling

Integration ...

Data Assessment

/Improvement

APIs, Querying, Data Management, etc.

DaaS design & implementation –

not just „functional“ aspects (2)

ASE WS 2012 26

Understand the DaaS ecosystem

Specifying, Evaluating and Provisioning Data

concerns and Data Contract

In follow-up

lectures

WHAT ARE OTHER PATTERNS

IN „TURNING DATA TO

DAAS“?

Discussion time

ASE WS 2012 27

DaaS ecosystems

ASE WS 2012 28

Data Assessment and Enrichment

Marco Comerio, Hong Linh Truong, Carlo Batini, Schahram Dustdar: Service-oriented data quality engineering and

data publishing in the cloud. SOCA 2010: 1-6

Examples of service units in DaaS

ecosystems

ASE WS 2012 29

Platforms/services Capabilities

Strikeiron clean, verify and validate data.

Jigsaw clean, verify and validate

business contact.

PostcodeAnywhere capture, clean, validate

and enrich business data.

Trillium Software Quality clean and standardize data

Uniserv Data Quality Solution X profile and clean data

Adeptia Integration Solution integrate data

Marco Comerio, Hong Linh Truong, Carlo Batini, Schahram Dustdar: Service-oriented data quality engineering and

data publishing in the cloud. SOCA 2010: 1-6

DaaS ecosystem –

profiling/enriching example

ASE WS 2012 30

http://www.strikeiron.com/

Cloud-based conceptual architecture

for data quality and enrichment

ASE WS 2012 31

Marco Comerio, Hong Linh Truong, Carlo Batini, Schahram Dustdar: Service-oriented data quality engineering and

data publishing in the cloud. SOCA 2010: 1-6

WHY DO YOU NEED TO STUDY

DAAS CONCEPTS, DESIGN

AND IMPLEMENTATION, AND

ECOSYSTEMS?

Discussion time

ASE WS 2012 32

Exercises

Read mentioned papers

Check characteristics, service models and

deployment models of mentioned DaaS (and

find out more)

Identify services in the ecosystem of some DaaS

Write small programs to test public DaaS, such

as COSM/Pachube, Microsoft Azure and

Infochimps

Turn some data to DaaS using existing tools

ASE WS 2012 33

34

Thanks for your attention

Hong-Linh Truong

Distributed Systems Group

Vienna University of Technology

truong@dsg.tuwien.ac.at

http://www.infosys.tuwien.ac.at/staff/truong

ASE WS 2012