Analyzing and Specifying Concerns for

DaaS

Hong-Linh Truong

Distributed Systems Group,

Vienna University of Technology

truong@dsg.tuwien.ac.at http://www.infosys.tuwien.ac.at/staff/truong

1 ASE WS 2012

Advanced Services Engineering,

WS 2012, Lecture 4

Outline

What are data concerns and why their are

important

Issues in DaaS concerns

Analysis and specification of DaaS concerns

ASE WS 2012 2

....

What are data concerns?

data DaaS .... data assets

APIs, Querying, Data Management, etc.

Located

in US?

free?

price?

redistribution? Service

quality?

3 ASE WS 2012

Quality of data? Privacy

problem?

....

DaaS Concerns

ASE WS 2012 4

data DaaS .... data assets

Data

concerns

Quality of

data Ownership

Price License ....

APIs, Querying, Data Management, etc.

DaaS concerns include QoS, quality of data (QoD),

service licensing, data licensing, data governance, etc.

Why DaaS/data concerns are

important?

Too much data returned to the

consumer/integrator

Results are returned without a clear usage and

ownership causing data compliance problems

5

Ultimate goal: to provide relevant data with

acceptable constraints on data concerns

ASE WS 2012

Example: Mashup (1)

Composition of Yahoo! Boss News Search,

Google News Search , and Flickr

recent news and high-qualified images, but free-

of charge, related to "Haiti earthquake"

6

Hong Linh Truong, Marco Comerio, Andrea Maurino, Schahram Dustdar, Flavio De Paoli, Luca Panziera: On

Identifying and Reducing Irrelevant Information in Service Composition and Execution. WISE 2010: 52-66

ASE WS 2012

7

Example: Mashup (2)

8

If the composer is aware of context

and quality parameters

Possible mappings of context and quality

requirements

but it is a tedious task and hard to be automated and we

are not sure we have a correct mapping.

ASE WS 2012

9

Example: open data (1)

Retrieve big datasets from RESTful services for further

extraction, transform or data composition activities

http://www.undata-api.org/

ASE WS 2012

10

Example: open data (2)

Example: study the population growth and

literacy rate from 1990-2009 for all countries in

the world

Without QoD: get datasets and perform mashup

ASE WS 2012

11

Example: open data (2)

With QoD support:

Population annual growth rate (percent):

dataelementcompleteness= 0.8654708520179372,

datasetcompleteness=0.7356502242152466;

Adult literacy rate (percent):

dataelementcompleteness=0.5874439461883408

datasetcompleteness=0.04349775784753363

Should we retrieve the data and perform data

composition?

ASE WS 2012

Countries\Year 1990 ... 2009

1

...

223

223

elements

12

Example: smart environments

Smart environments with several low level sensors:

Recognize human activities: idle, relaxing, and cleaning

up,

Provide context information for adaptive service

discovery and execution

E.g., FP7 SM4All, FP7 EU OPPORTUNITY

Virtual Sensor-as-a-Service provides human activities

ASE WS 2012

13

Example: smart environments (2)

PoC: Probability of Correctness

QoC: Quality of Context

VSS: Virtual Sensor Service

CMS: Context Management Service

CCS: Context Consumer Service

AC: Appliances Control (AC)

AM: Ambiance Management

Atif Manzoor, Hong Linh Truong, Christoph

Dorn, Schahram Dustdar: Service-centric

Inference and Utilization of Confidence on

Context. APSCC 2010: 11-18

ASE WS 2012

WHAT ARE OTHER CASES

WHERE DAAS CONCERNS

ARE IMPORTANT FOR?

Discussion time

ASE WS 2012 14

Issues on DaaS concerns (1)

DaaS concern models

Unstructured description of context, QoS and

quality of data (QoD)

Different specifications and terminologies

Mismatching semantics of information about

services and data concerns

15 ASE WS 2012

Issues on DaaS concerns (2)

DaaS APIs

No/Limited description of data and service

usage

No API for retrieving quality and context

information

No quality and context information associated

with requested data

16 ASE WS 2012

Issues on DaaS concerns (3)

Evaluation techniques

Missing evaluation of compatibility of context

and concerns for multiple DaaS and data

assets

Missing evaluation techniques to filter

large/irrelevant data quantity

17

Require a „holistic integration“ of information models,

APIs and evaluation techniques for DaaS concerns!

ASE WS 2012

18

Solutions needed

ASE WS 2012

Developing techniques for context and DaaS concerns evaluation

On-the-fly data concerns evaluation

Concerns compatibility evaluation and composition

Developing context and DaaS concerns that can be accessed via open APIs

APIs extension External DaaS information service

Developing meta-model and domain-dependent semantic representations for quality and context information specifications

Reconciliation of DaaS concern terms

Linked DaaS concerns models

WHY CONTEXT IS

IMPORTANT?

Discussion time

ASE WS 2012 19

DaaS concerns analysis and

specification

Which concerns are important in which

situations?

How to specify concerns?

20 ASE WS 2012

Hong Linh Truong, Schahram Dustdar On analyzing and specifying concerns for data as a service. APSCC 2009: 87-

94

The importance of concerns in

DaaS consumer‘s view – data

governance

ASE WS 2012 21

Important factor, for example, the security and

privacy compliance, data distribution, and auditing

Storage/Database

-as-a-Service data DaaS

Data governance

The importance of concerns in DaaS

consumer‘s view – quality of data

Read-only DaaS

Important factor for the

selection of DaaS.

For example, the

accurary and

compleness of the data,

whether the data is up-to-

date

CRUD DaaS

Expected some support

to control the quality of

the data in case the data

is offered to other

consumers

22 22 ASE WS 2012

The importance of concerns in

DaaS consumer‘s view– data and

service usage

Read-only DaaS

Important factor, in

particular, price, data

and service APIs

licensing, law

enforcement, and

Intellectual Property

rights

CRUD DaaS

Important factor, in

paricular, price, service

APIs licensing, and law

enforcement

ASE WS 2012 23

The importance of concerns in

DaaS consumer‘s view – QoD

Read-only DaaS

Important factor, in

particular availability and

response time

CRUD Daas

Important factor, in

particular, availability,

response time,

dependability, and security

ASE WS 2012 24

The importance of concerns in DaaS

consumer‘s view– service context

Read-only DaaS

Useful factor, such as

classification and service

type (REST, SOAP),

location

CRUD DaaS

Important factor, e.g.

location (for regulation

compliance) and versioning

ASE WS 2012 25

WHAT ARE OTHER

IMPORTANT ISSUES? ADD

YOUR FINDING!

Discussion time

ASE WS 2012 26

Conceptual model for DaaS

concerns and contracts

27 ASE WS 2012

Capability concerns

Data Quality capabilities Based on well-established research on data quality

Timelineness, uptodate, free-of-error, cleaning, consistency,

completeness, domain-specific metrics, etc.

We mainly support the specification of QoD metrics for the whole

DaaS but possible to extend to the service operation level

Data Security/Privacy capabilities Data protection within DaaS, e.g. encryption, sensitive data

filtering, and data privacy

Many terms are based on the W3C P3P

28 ASE WS 2012

Capability concerns (2)

Auditing capabilities

Logging, reporting (e.g., daily, weekly, and monthly),

and warning

Support system maintenance, SLA monitoring, billing,

and taxation

Data lifecycle

Backup/recovery, distribution (e.g., a service is in

Europe but data is stored in US), and disposition

Support system maintenance but also regulation on

data

29 ASE WS 2012

Capability concerns (3)

Data and service license

Usage permission: for data (distribution, transfer,

personal use, etc.) and for service APIs (adaptation,

composition, derivation, etc.)

We utilize some terms from ODRL/ODRL-S

Copyrights

Liability: e.g., who is reponsible for the loss due to a

network disruption?

Law enforcement (e.g., US or European court)

Domain specific Intellectural property rights

30 ASE WS 2012

Data source concerns

A DaaS may utilize data from many sources.

Similar DaaSs may utilize data from the same source

Data source properties

Name: e.g. ddfFlus or DataFlux

Size

Timespan: the duration of collected data,

Update Frequency: how offen the data is updated

etc

31 ASE WS 2012

Service context concerns

Location:

Selecting a DaaS in Amazon US Zone or European Zone?

Service Type: REST or SOAP?

Level of Service

Service Classification

Based on UNSPSC Code Classification Services

Data Classification

Service/data versioning

32 ASE WS 2012

33

XML Diagram for the DaaS

capability specification

33 ASE WS 2012

34

XML Diagram for DaaS specification

ASE WS 2012

35

From capability/context to

DaaS contract

35

Search

properties of

DaaSs

Define and

negotiate contract

terms

Contracts

DaaS Capabilities,

Context, Data

Source

Consumer-specific

concerns

A DaaS contract includes a set of generic, data-

specific and service-specific conditions established

based on concerns

ASE WS 2012

Recall -- stakeholders in data

provisioning

ASE WS 2012 36

Data

Data Provider

• People (individual/crowds/organization)

• Software, Things

Service Provider

• Software and people

Data Consumer

• People, Software, Things

Data Aggregator/Integrator

• Software

• People + software

Data Assessment

• Software and people

37

Populating DaaS concerns

DaaS

Concerns

evaluate, specify,

publish and manage

specify, select,

monitor, evaluate

monitor and

evaluate

The role of stakeholders in the most trivial view

Data Aggregator/Integrator

Data Consumer

Data Assessment

Service Provider

Data Provider

ASE WS 2012

Support DaaS concerns selection

38

Service Information

Management

Service

SEMF-based

information, including

concerns

DeXIN

Data

Consumer

External

sources

SECO2

1. Muhammad Intizar Ali, Reinhard Pichler, Hong Linh Truong, Schahram Dustdar: Data Concern Aware Querying

for the Integration of Data Services. ICEIS (1) 2011: 111-119

2. Marco Comerio, Hong Linh Truong, Flavio De Paoli, Schahram Dustdar: Evaluating Contract Compatibility for

Service Composition in the SeCO2 Framework. ICSOC/ServiceWave 2009: 221-236

ASE WS 2012

39

Implementation (1)

Check http://www.infosys.tuwien.ac.at/prototyp/SOD1/dataconcerns

40

Implementation (2)

Data privacy concerns are annotated with WSDL

and MicroWSMO

41

Implementation (3)

Joint work with

http://infochimps.org/datasets/twitter-haiti-earthquake-data

Michael Mrissa, Salah-Eddine Tbahriti, Hong Linh

Truong: Privacy Model and Annotation for

DaaS. ECOWS 2010: 3-10

Some Studies

We are not aware of any provider that publishes

DaaS‘s concerns in a well-defined form

Mainly in HTML

Our studies examines the description of DaaSs

Enterprising computing

StrikeIron, Xignite, serviceobjects.NET, WebserviceX,

XWebServices, AERS, Amazon

E-science

GBIF (Global Biodiversity Information Facility), EBI

(European Bioinformatics Institute) Web Services,

EMBRACE Service Registry, and BioCatalogue

42 ASE WS 2012

Concerns in HTML descriptions

29 services from 7 providers, most are SOAP-

based

43

Com

ple

teness

Upto

date

Corr

ectn

ess

Cle

anin

g

Sta

ndard

outp

ut

Priva

cy

Loggin

g

Report

ing

Warn

ing

Backup

Response T

ime

Ava

ilabili

ty

Netw

ork

Late

ncy

Packet

Loss

Netw

ork

Security

Price M

odel

Serv

ice C

redit

Usage P

erm

issio

n

Copyright

Lia

bili

ty

Law

Enfo

rcem

ent

Dom

ain

-specifi

c IP

R

Location

Serv

ice T

ype

Data

Cla

ssifi

cation

Data

Sourc

e N

am

e

Data

Sourc

e S

ize

Data

Sourc

e U

pdate

Fre

q.

0

5

10

15

20

25

30

35

Not mentioned/clear

Mentioned

Hong Linh Truong, Schahram Dustdar On analyzing and specifying concerns for data as a service. APSCC 2009: 87-

94

ASE WS 2012

Concerns of DaaSs in E-science

Service Registries DQ QoS Business Licensing

Ownership Usage

permission

GBIF No No No unstructured unstructured

EBI Web Services No No No No No

EMBRACE Service

Registry

No No No No No

BioCatalogue No No unstructured unstructured unstructured

From the DaaS description point of view

Hong Linh Truong, Schahram Dustdar On analyzing and specifying concerns for data as a service. APSCC 2009: 87-

94

ASE WS 2012 44

WHAT CAN WE DO MORE

WITH INFORMATION ABOUT

DAAS CONCERNS?

Discussion time

ASE WS 2012 45

Exercises

Read mentioned papers

Visit DaaS mentioned in previous lectures

Analyze existing DaaS concerns

Examine how they specify and publish concerns

Investigate possible concerns when merging

data from different types of DaaS

Open government data and near-realtime data from

sensors

ASE WS 2012 46

47

Thanks for your attention

Hong-Linh Truong

Distributed Systems Group

Vienna University of Technology

truong@dsg.tuwien.ac.at

http://www.infosys.tuwien.ac.at/staff/truong

ASE WS 2012