tuw- 184.742 data as a service – concepts, design & implementation, and ecosystems
Post on 22-Apr-2015
582 Views
Preview:
DESCRIPTION
TRANSCRIPT
Data as a Service – Concepts, Design &
Implementation, and Ecosystems
Hong-Linh Truong
Distributed Systems Group,
Vienna University of Technology
truong@dsg.tuwien.ac.at http://www.infosys.tuwien.ac.at/staff/truong
1 ASE WS 2012
Advanced Services Engineering,
WS 2012
Outline
Data provisioning and data service units
Data-as-a-Service concepts
DaaS design and implementation
DaaS ecosystems
ASE WS 2012 2
Data versus data assets
ASE WS 2012 3
Data
Data Assets
Data management
and provisioning
Data concerns
Data collection,
assessment and
enrichment
Data provisioning activities and
issues
ASE WS 2012 4
Collect
• Data sources
• Ownership
• Quality assessment and enrichment
Store
• Query and backup capabilities
• Local versus cloud, distributed versus centralized storage
Access
• Interface
• Public versus private access
• Access granularity
• Pricing and licensing model
Utilize
• Alone or in combination with other data sources
• Redistribution
Non-exhausive list! Add your own issues!
Stakeholders in data provisioning
ASE WS 2012 5
Data
Data Provider
• People (individual/crowds/organization)
• Software, Things
Service Provider
• Software and people
Data Consumer
• People, Software, Things
Data Aggregator/Integrator
• Software
• People + software
Data Assessment
• Software and people
Recall – Service Unit
ASE WS 2012 6
Service model
Unit Concept
Service unit
„basic
component“/“basic
function“ modeling
and description
Consumption,
ownership,
provisioning, price, etc.
What about service units providing data?
Data service unit
ASE WS 2012 7
Service model
Unit Concept
Data service
unit
Data
Can be used for private
or public
Can be elastic or not
Data service units in clouds/internet
Provide data capabilities rather than provide
computation or software capabilities
Providing data in clouds/internet is an increasing
trend
In both business and e-science environments
Bio data, weather data, company balance
sheets, etc., via Web services
8 ASE WS 2012
Data service unit
9
Data service units in
clouds/internet
data
Internet/Cloud
Data service unit
People
data
Data service unit
Things
ASE WS 2012
data data
SO DATA SERVICE UNIT IS
BIG OR SMALL? PROVIDING
REALTIME OR STATIC DATA?
Discussion time
ASE WS 2012 10
11
NIST Cloud definitions
“This cloud model promotes availability and is
composed of five essential characteristics,
three service models, and four deployment
models.”
ASE WS 2012
Source: NIST Definition of Cloud Computing v15, http://csrc.nist.gov/groups/SNS/cloud-computing/cloud-def-v15.doc
Data as a Service -- characteristics
On-demand self-service
Capabilities to provision data at different granularities
Resource pooling
Multiple types of data, big, static or near-realtime,raw data and
high-level information
Broad network access
Can be access from anywhere
Rapid elasticity
Easy to add/remove data sources
Measured service
Measuring, monitoring and publishing data concerns and usage
ASE WS 2012 12
Built atop NIST‘s definition
Data-as-a-Service – service models
Data as a Service – service models
and deployment models
ASE WS 2012 13
Storage-as-a-Service
(Basic storage functions)
Database-as-a-Service
(Structured/non-structured
querying systems)
Data publish/subcription
middleware as a service
Sensor-as-a-Service
Private/Public/Hybrid/Community Clouds
deploy
Examples of DaaS
ASE WS 2012 14
WHAT ELSE DO YOU THINK
CAN BE INCLUDED INTO DAAS
MODELS?
Discussion time
ASE WS 2012 15
DaaS design & implementation –
APIs
Read-only DaaS versus CRUD DaaS APIs
Service APIs versus Data APIs
They are not the same wrt concerns
SOAP versus REST
ASE WS 2012 16
Example: infochimps
DaaS design & implementation –
service provider vs data provider
The DaaS provider is separated from the data
provider
17
DaaS
Consumer
DaaS
Sensor
DaaS
Consumer DaaS provider Data
provider
ASE WS 2012
Example: DaaS provider =! data
provider
18
DaaS design & implementation –
structures
DaaS and data providers have the right to
publish the data
ASE WS 2012 19
DaaS
• Service APIs
• Data APIs for the whole resource
Data Resource
• Data APIs for particular resources
• Data APIs for data items
Data Items
• Data APIs for data items
Three levels
20
DaaS design & implementation –
structures (2)
Data
items
Data
items
Data
items
Data resource
Data
assets
Data resource Data resource
Data resource Data resource
Consumer
Consumer
DaaS
ASE WS 2012
DaaS design & implementation –
patterns for „turning data to DaaS“ (1)
ASE WS 2012 21
DaaS data Build Data
Service
APIs
Deploy
Data
Service
Examples: using WSO2 data service
Storage/Database
-as-a-Service
DaaS design & implementation –
patterns for „turning data to DaaS“ (2)
ASE WS 2012 22
data
Examples: using
Amazon S3
DaaS
Storage/Databa
se/Middleware
DaaS design & implementation –
patterns for „turning data to DaaS“ (3)
ASE WS 2012 23
data
Examples: using
COSM/Pachube
Things
One thing 10000... things
DaaS
Storage/Database/
Middleware
DaaS design & implementation –
patterns for „turning data to DaaS“ (4)
ASE WS 2012 24
data
Examples: using Twitter
People DaaS
....
DaaS design & implementation –
not just „functional“ aspects (1)
ASE WS 2012 25
data DaaS .... data assets
Data
concerns
Quality of
data Ownership
Price License ....
Enrichment Cleansing
Profiling
Integration ...
Data Assessment
/Improvement
APIs, Querying, Data Management, etc.
DaaS design & implementation –
not just „functional“ aspects (2)
ASE WS 2012 26
Understand the DaaS ecosystem
Specifying, Evaluating and Provisioning Data
concerns and Data Contract
In follow-up
lectures
WHAT ARE OTHER PATTERNS
IN „TURNING DATA TO
DAAS“?
Discussion time
ASE WS 2012 27
DaaS ecosystems
ASE WS 2012 28
Data Assessment and Enrichment
Marco Comerio, Hong Linh Truong, Carlo Batini, Schahram Dustdar: Service-oriented data quality engineering and
data publishing in the cloud. SOCA 2010: 1-6
Examples of service units in DaaS
ecosystems
ASE WS 2012 29
Platforms/services Capabilities
Strikeiron clean, verify and validate data.
Jigsaw clean, verify and validate
business contact.
PostcodeAnywhere capture, clean, validate
and enrich business data.
Trillium Software Quality clean and standardize data
Uniserv Data Quality Solution X profile and clean data
Adeptia Integration Solution integrate data
Marco Comerio, Hong Linh Truong, Carlo Batini, Schahram Dustdar: Service-oriented data quality engineering and
data publishing in the cloud. SOCA 2010: 1-6
DaaS ecosystem –
profiling/enriching example
ASE WS 2012 30
http://www.strikeiron.com/
Cloud-based conceptual architecture
for data quality and enrichment
ASE WS 2012 31
Marco Comerio, Hong Linh Truong, Carlo Batini, Schahram Dustdar: Service-oriented data quality engineering and
data publishing in the cloud. SOCA 2010: 1-6
WHY DO YOU NEED TO STUDY
DAAS CONCEPTS, DESIGN
AND IMPLEMENTATION, AND
ECOSYSTEMS?
Discussion time
ASE WS 2012 32
Exercises
Read mentioned papers
Check characteristics, service models and
deployment models of mentioned DaaS (and
find out more)
Identify services in the ecosystem of some DaaS
Write small programs to test public DaaS, such
as COSM/Pachube, Microsoft Azure and
Infochimps
Turn some data to DaaS using existing tools
ASE WS 2012 33
34
Thanks for your attention
Hong-Linh Truong
Distributed Systems Group
Vienna University of Technology
truong@dsg.tuwien.ac.at
http://www.infosys.tuwien.ac.at/staff/truong
ASE WS 2012
top related