Uptake of Service Oriented

Architecture in Statistical Agencies -

Are We Really so Different?

Matjaz Jug

SOA Definition

• The Open Group describes Service Oriented

Architecture (SOA) as a:

– “style of IT architecture that delivers agility and

Boundaryless Information Flow™. It is deployed

on an increasing scale in enterprises today.”

SOA Meta model, Linthicum Group, 2007

Data Abstraction

Data Data

Data Services/Messaging

Legacy Legacy

Services

Process/Orchestration

Monitoring/Event Management

Gove

rn

an

ce

Rep

Se

cu

rity

New Services

http://weblog.infoworld.com/realworldsoa/archives/Full%20Contact%20SOA.ppt

SOA Benefits

• Increased agility: organisations should be

able to more quickly respond to changes in

business process and external environment.

• Reduction of development cost trough reuse:

new IT systems should be able to leverage

the most readily available code and services

from across organization and externally.

• Better possibilities for integration using

loosely coupled framework and orchestration.

• Configuration rather than programming

Why Should We Care?

• Agility: transformational changes like shift

towards the increased use of administrative

data, more automated data processing etc.

• Reuse: reducing high cost of development and

maintenance of statistical production systems.

• Integration: need for integration of statistical

tools and many legacy application assets.

• Configuration: response to frequent changes in

data sources, questionnaires, methodology

and classifications.

SOA Maturity Models

Source: http://www.sonicsoftware.com/solutions/service_oriented_architecture/soa_maturity_model/index.ssp

Source: Inaganti and Sriram.

www.bptrends.com

SOA Maturity Models

Source: http://adtmag.com/articles/2007/06/18/soa-uptake-still-early-analysts-say.aspx

SOA Uptake Still 'Early,' Analysts Say

By Kurt Mackie

06/18/2007

A report from Research 2.0 describes the current use

of service-oriented architecture (SOA) as

experimental. The research firm's report of May 31,

2007 predicts that SOA will be embraced as

mainstream technology by the year 2015.

2007

Source: http://www.gartner.com/DisplayDocument?ref=g_search&id=750742

User Survey Analysis: SOA and Application

Development and Integration Trends, Worldwide,

2008

3 September 2008

Teresa Jones

Gartner conducted a user survey in 2008 regarding

service-oriented architecture and application

development trends. Survey results reveal a strong

uptake of SOA around the world.

2008

January 05, 2009

SOA is Dead; Long Live Services

Blogger: Anne Thomas Manes

Obituary: SOA

SOA met its demise on January 1, 2009,

when it was wiped out by the catastrophic

impact of the economic recession.

SOA is survived by its offspring:

mashups, BPM, SaaS, Cloud

Computing, and all other

architectural approaches

that depend on “services”.

Source: http://apsblog.burtongroup.com/2009/01/soa-is-dead-long-live-services.html

2009

Early Uptake in NSOs

• Statistics Finland (2004): MetaAPI which aimed to

develop application programming interface built on

top of the existing metadata management systems.

• ONS (2004): Redevelopment project, important part

was SOA-based integration between many different

statistical tools and data and metadata repositories.

• Statistics New Zealand (2005): Business model

Transformation Strategy (BmTS) with the aim to

develop new generic business process model. Two

key architectural elements were the Ten Component

Information Model and the utilisation of a SOA.

http://www-03.ibm.com/industries/travel/us/detail/resource/I280982G39160Z17.html

Service orientation in the airline industry

Address irregular operations flexibly and efficiently

Weather delays. Lost baggage. Maintenance downtime. These

irregular events can result in unhappy customers, higher costs

and—potentially—lost revenue. To improve cross-enterprise

collaboration, enable better-informed decisions and provide

consistency across the enterprise during irregular operations, you

must provide access to common information. Service orientation

can help by breaking everyday business applications into

individual business tasks, called services. These services can then

be shared to provide timely information to agents and customers,

enhance responsiveness and improve customer satisfaction.

SOA in the Airline Industry

SOA in the Airline Industry

Air NZ CEO chides IBM over yesterday's outage

Internal email shows frustration at crash that affected

check-in, bookings and call centre

By Randal Jackson Wellington | Monday, 12 October, 2009

Air New Zealand outsourced its

mainframe to IBM in 1997.

Air New Zealand was to meet IBM today over the crash, which

took down airport check-in systems, as well as online bookings

and call centre systems about 9.30am yesterday, affecting more

than 10,000 passengers and throwing airports into chaos.

http://computerworld.co.nz/news.nsf/news/A4E6A75BBC7950A4CC25764C007A8E41

Situation in Statistical Organizations

• Many lessons learnt and challenges were reported

from early adopters.

• Even now there are not a lot statistical organisations

implementing SOA on a large scale.

• It is actually quite hard to find statistical organisation

with maturity level higher than 2.

• We are apparently “behind” comparing to some other

sectors like Airline Industry.

• WHY? Are we really so different?

• Or is it maybe only the matter of time?

Other Paradigms we Can Learn from

Data Warehousing

• Introduced in 80’s (but

used in statistical offices

even earlier - Austria,

Sweden)

• The entire statistical

office can be a

warehouse?

• Technology is mature and

widely adopted with many

commercial tools

available

Metadata-driven Approach

• Introduced in 90’s (but

used in statistical offices

even earlier?)

• End-to-end metadata-

driven process?

• Hard to find commercial

solution although it exists

for some areas (ERP)

• Data warehouse is

usually part of the

metadata-driven solution

Benefits

DW MD

SOA

Data access

& analysisInternal

process

Metadata

management

Reuse/

agilityIntegration

Externalisation

Learning from DW and MD projects –

Why so Slow Uptake?1. High degree of organisational change is required

which is usually slow process.

2. It is difficult to establish new governance.

3. New architecture usually requires complete

replacement of legacy application assets portfolio.

4. Software development isn’t a core business of

statistical organisations.

5. Common challenge organisations often face

involves effectively managing metadata.

6. Lack of standardisation – it appears every new

paradigm requires more of it.

Standardisation

Service Management

Metadata management

Data Management

In Some Areas We are Different!

• Many semantically diverse data structures.

• Frequent change in data structure, sources,

questionnaires.

• Specific requirements like data confidentiality.

• Many stove-piped legacy application assets

• Mainly non-transactional processing

• End-user processing environments

How to Start? Areas Where SOA

Can Deliver Significant Value• Metadata services: a good candidate for

reuse in many stovepipe and corporate

applications.

• Statistical tools/components: making them

more interoperable using service interface

would significantly improve the possibilities to

integrate them in different IT environments

and therefore increase their shared usage

and collaboration.

Why is Collaboration so Important?

IBM's SPSS Buy Threatens Oracle, SAP And SAS

July 29, 2009

By Jeffrey Burt

Buying its data mining partner, IBM issues a challenge to its

rivals Oracle, SAP and SAS to up their game

IBM’s proposed $1.2 billion acquisition of SPSS is the vendor's most

significant move towards giving customers tools to collect and use

data, since it bought Cognos in 2007.

It puts software rivals such as Oracle, SAP and SAS Institute on

notice that IBM is pulling together a substantial package of solutions

in this area, and probably is a precursor to a host of similar

acqusitions by other vendors, according to analysts.http://www.eweekeurope.co.uk/knowledge/ibm-s-spss-buy-threatens-oracle--sap-and-sas--1496

Why is Collaboration so Important?

We will not be able to purchase our statistical

applications.. at least not very soon

NEW IMPUTATION

MODULE

DATA CONFIDENTIALITY

OUT OF THE BOX

CAPI/CATI/CAWI SUPPORTEDNEW REALEASE WITH

SELECTIVE EDITING

The Next Great IT Paradigm is…

Software as a Service

(SaaS) delivered via

Internet (cloud

computing)

… it will require integration between internal

and external systems. It is however not very

surprising that the success of this integration and

therefore uptake of SaaS will rely on services.

it is not a silver bullet AND

We are not so different… at least not among ourselves.

Questions?