presentatie 20071121 dutch railways and soa avans (1x90min) v1.0

43
1 Dutch Railways and SOA Jack van Hoof JvH-Y2K+7 v1.0

Upload: jack541108

Post on 11-Feb-2017

712 views

Category:

Business


4 download

TRANSCRIPT

Page 1: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

1

Dutch Railways and SOA

Jack van Hoof

JvH-Y2K+7 v1.0

Page 2: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

2

This presentation

1. Who am I

2. What is Dutch Railways

3. Vision of Dutch Railways on the application landscape

4. Application level: Role of Web services technology at Dutch Railways

5. Infrastructure level: Role of the ESB as a Web services platform at Dutch Railways

6. Our view on the innovation roadmap

- High level, no technical details -

Page 3: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

3

Who am I? Jack van Hoof…

The 90-s: Dutch Defence (The Hague)

• Business consultancy

• System development

• Leading developmentteams

The 80-s: IBM (Amsterdam/Uithoorn)

• Leading analyst of development teams

• Programming/design/analyses

The 21st century: Dutch Railways (Utrecht)

• IT-architect – Application Integration, SOA, ESB

Page 4: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

4

http://soa-eda.blogspot.com

- Thoughts on SOA and EDA -

My weblog

Page 5: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

5

What is Dutch Railways?

• Passengers transportation by train (no freight transport)

• Owner of railway stations (maintenance and operation)

• Owner of buildings and terrains (biggest real estate owner in The Netherlands)

• Time tables

• Maintenance and repair of trains (not only Dutch Railways trains)

• Commerce (price of the tickets)

• Training of drivers and conductors

Page 6: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

6

How big is Dutch Railways?

• Number of coaches: 3000

• Number of stations: 400

• Number of travellers per day: 1.000.000

• Number of traveller-kilometers per day: 45.000.000 (1.000 x around the world)

• Number of “front-liners” (drivers, conductors, station-personel): 10.000

• Total number of employees: 20.000

Page 7: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

7

Our Vision

Page 8: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

8

We strive for “new” technology

Application infrastructure• Enterprise Service Bus • Portals

Open standards• XML• Web services: SOAP / WSDL / WS-*• WSRP • JSR168

Architectural approaches• SBI (Service Based Integration) • SOA (Service Oriented Architecture)• EDA (Event Driven Architecture)

Driver: Not cost reduction, but necessary to survive

Page 9: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

9

Why should we strive for “new” technology?

The whole IT-world is converging into this direction

• Market analysts (Gartner): SOA / EDA

• Software vendors: XML / WSDL / SOAP / WS-*

• Organizations: Flexible business-to-business value chains with use of uniform/standard technology

Ignoring these trends means getting away from your partners

• Modern software relies on modern technology (expensive to adapt new software to legacy)

• Environment demands uniform connectivity with regard to B2B value chains: take-it or leave-it

Because everone does…

so if we don’t want to get isolated we have to join

Driver:

Not cost reduction, but necessary to survive

Innovation is no option, but a must in the current era

Page 10: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

10

Does Dutch Railways have an SOA?

No…

But we will, as everyone!• All software suppliers are service enabling their application interfaces

• All software tools will enforce Web services technologies

• You can’t (in future) implement ERP (SAP) without implementing SOA (SOA out-of-the-box)

• Business partners will force us to communicate via standard technology (web services)

• SaaS providers will (in future) use SOA to allow for customized business process definitions

accross multiple providers

We anticipate by service enabling our legacy,

and putting an adquate infrastructure in place

Page 11: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

11

Current situation: 500 “known” applications

Every application has own solution/technology for:

•Access control

•User interface mechanism

•Data storage

•Application server

•Interface-mechanisms•shared databases•file shares (accross domains)•Mail•FTP•remote calls

Monolithic applications

User interface and access control

Business logic

Data storage

Interface

No structure!

No overview!

Page 12: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

12

Getting structure: separation of concerns

Generic solutions for: •Application access (Portal)

•Data storage (DBMS, datawarehouse, backup/recovery)

•Standardized interfaces (ESB, secured, reliable)

•Application platform (application servers)

Business logicservices

Acc

ess

serv

ices

Exchange area

Dat

a st

orag

ese

rvic

es

User interface and access control

Business logic

Data storage

Interface

Interaction based on open standards:

XML, SOAP, WSDL, JSR168, WSRP, WS-*

Effects:

•Better manageable

•Consistency (data, user interfaces, business logic)

•Cheaper deployments (one-time deployment)

•More flexible to change

Page 13: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

13

One virtual Enterprise application

Business logic services

ERP

Acc

ess

serv

ices

Exchange area

Dat

a st

orag

e se

rvic

es

COTS

Homemade Legacy

BusinessProcess

Management

BusinessActivity

Monitoring

(Complex)Event

processing

All business logic implementations must fit within framework:

•ERP - Enterprise oriented applications like SAP•COTS - Function oriented packages•Home made software•Legacy - Old fashioned monolithic applications

Flexibility and uniformity, but also basis for:

•Business Process ManagementFlexible definition of processes outside applications

•Business Activity Monoriting Real-time insight in operational business state

•Event processingReact proactive to potential bottle-necks

Page 14: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

14

Business logic services

ERP

Acc

ess

serv

ices

Exchange Area

Dat

a st

orag

e se

rvic

es

COTS

Home made Legacy

BusinessProcess

Management

BusinessActivity

Monitoring

(Complex)Event

Processing

Web

2.0

clie

nts

(web

bro

wse

rs)

AJAX mashups

Portal with portlets

Databases

SOA and EDA

ESB

Implementation technologies

Page 15: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

15

Two important challenges

Heterogenous technologies within Dutch Railways• Programming languages• Databases• Operating systems• Hardware • Devices

• Gates at stations• Camera’s• Railpockets (PDA’s for front-liners)• Information displays

Heterogenous environments of Dutch Railways• Geographical locations

• Stations• (Moving) trains• Data Centers

• External partners• ProRail (rail infrastructure)• Other transportation companies (time tables)• Service providers (cleaning, maintenance)• Retailers

1

2

Putting together:

Page 16: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

16

Current market trends are helping us

Our needs

• Standards to harmonize different technologies

• Infrastructure to virtualize different locations

Market trends

• Suppliers strive for uniformity and standardization - Web services

• Generic distributed integration technologies are coming to market - ESB

Page 17: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

17

We go for Web services and ESB

By using standard Web services technology everything can be connected with everything

Enterprise Service Bus (ESB) is our deployment platform for Web services technology

Application level

Infrastructure level

Page 18: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

18

The application level

Page 19: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

19

Application innovation: 3 different focus points

• Service Based Integration (SBI): focus on monolithic applications – wrapping interfaces

• Service Oriented Architecture (SOA): focus on modular application construction - “call” services

• Event-driven Architecture (EDA): focus on business events – publish and consume messages

Page 20: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

20

SBI: focus on monolithic application

New

Connect monolithic applications (legacy, COTS, ERP) using

Web services technology

to harmonize different technologies

Application B Application C

Application A

WebServices

Keywords: Legacy, stove-pipes, packaged software (COTS, ERP)

Application B Application C

Application A

Old

Looking for common technologies to communicate

e.g: B with C via fileshare on server of A

Page 21: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

21

SOA: focus on modular application construction

• Reusable components deliver services to each other• Process definitions outside the components (Process Manager)• Ultimate: one enterprise wide box of building blocks

B1

B2

C3

C1

C2

A3

SOA

A1

A4 B3

A2

Keywords: Strong cohesion, command-and-control, reuse, functional decomposition

Calling services using Web services technology

Page 22: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

22

EDA: focus on event messaging

Events

Web Services

(SOAP)

Applications (legacy, SOA’s, workflows, transactions, processes, UI’s, portals, databases, gateways, devices)

Keywords: Loose coupling, linking autonomous processes, workflow

Publish and consume messages using Web services technology

Page 23: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

23

ESB

Simple example of EDA (illustrative)

Passenger xTrain

Routingtable

Gates at stationsa, b, c, d

Back-endsystems

SOAP SOAP SOAPSOAP SOAP

…buys ticket from A to B via Internet (business event)

…Determines stations on route A-D (enrichment)

…allow access to passenger x on date y

…register transaction

Passenger: xDate: yStations: a, b, c and d

Passenger: xDate: yRoute: A-D

Data-warehouse

SOAP

…logs data ibo analyses and rapporting

1 2 3 4 5Concurrent with: Concurrent with:

Page 24: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

24

Holistic approach at Dutch Railways

Access to old and packaged systems: Service based integration

Development of new systems (processes): Service oriented architecture

Connecting systems (processes) into chains: Event-driven architecture

Not subsequently, but all three concurrently!

And all with the same common technology: Web services

Page 25: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

25

Web Services: lubricating oil between old and new

Common technology allows for integration of legacy with SOA and EDA

Legacy Z

Legacy YS1

S4

S3

S5S2

SOA / EDASBI

Web services technology

“Old” “New”

Page 26: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

26

The infrastructure level- distributed web services platform -

Page 27: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

27

Web services alone are not enough

How do you deal with semantics?• Is “customer name” only last name, or first name AND last name …?

How do you deal with security?• Who is allowed to see or change a message? (confidentiallity, integrity)• Who is allowed to send or receive a message? (authorization, authentication)• How do you prove a message has been sent/received? (non-repudiation)

How do you deal with transactions?• How do you define a transaction and how will it be performed and rolled back?

How do you deal with workflow?• How is routing of messages defined and monitored? (intelligent routing when paths are blocked)

How do you deal with format transformation between sender and receiver?• How and where is data transformed during transport?

How do you deal with reliability?• How is guaranteed that a message reaches its destination?• How is guaranteed that a message is processed only once?• How is guaranteed that a message is processed in time?• Where does a published message persist before it is consumed?

?

?

?

?

?

?

Page 28: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

28

Supporting platform: The Enterprise Service Bus

Global Dataspace- ESB -

SOA(Process X)

Gateway

External systems Legacy(cusotm, package)

SOA (Process Y)

Package(COTS, ERP)

Enables support for:• Secured routing of messages • Guaranteed delivery of messages• Instant availability of messages within the enterprise• Semantic mapping between systems (CDM)• Format harmonisation between systems (CDM)• Explicit definition of busniness processes (BPM)• Insight in actual state of business processes (BAM)• Detect correlations between messages (CEP)• Access to services (SOA)

Publish and consume messages

Page 29: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

29

ESB

Reach of our ESB (future)

Stations

Trains

PDA’sData Centers

Railpockets

Gates

Partners

Gateway

All places where applications are running

Page 30: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

30

Appearance of our ESB

ESB

SAP-FINNetweaver XI

SAP-CRMNetweaver XI

SAP-HRNetweaver XI

IBM WebSphere(Corporate)

BEAWebLogic ESB

Partner Gateway

Etcetera(future)

BizTalk

Ddatagateway(MQ-Series)

External environment(business partners)

Remainingapplications

Page 31: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

31

ESB: Intelligent layer on the network

Connectivity

Enterprise Service Bus

Network

Security Reliability Transactions … User-defined

Host-config:DHCP

Name:DNS

Time:SNTP

Netwerkmngt:SNMP …

Technical infrastructure oriented services

Business oriented services

Messages flowing through ESB safely cross firewalls over HTTP port 80:

No connectivity issues anymore!

Page 32: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

32

ESB

ESB virtualizes location and technology

Model of application landscape

Location and technology virtualization

Connectivity

• Location and technology of applications are made invisible (unimportant) by the ESB • Visibility and overview of interaction between applications • Application landscape becomes manageable by visual tools

ESB: - Tools to get overview- Virtualizes locations and technology- Distributed presence at locations

Network:

- Technical connection between locations

Locations:

- Places where applications run

Page 33: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

33

Infrastructural characteristics of an “ideal” ESB

Based on open standards• Also internal mechanismes based on web services technology

• Makes interoperability with other ESB’s easy• Makes distributed architectures possible

Light weight• Adding functionality on demand over time when needed • Start on small scale and grow over time when needed

Distributed architecture• Intelligent components deployed near the endpoints (like agents)• Communicating components, no central hub processing (very scalable)• Only central rules repository; changes are published to local components

Page 34: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

34

Current technology trends

In a few years the network will be the ESB

The network (CISCO)

• Network devices (routers) are getting XML-aware

• Network decives are getting SOAP-aware

• Network devices will understand Web services standards (WS-*)

The ESB (IBM)

• XML-processors are getting build into appliances (hardware)

• SOAP-message security is getting managed by appliances

• SOAP-messages are filtered, blocked and routed by appliances

• Web services standards (WS-*) are getting implemented by appliances

Page 35: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

35

Innovation roadmap

Page 36: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

36

Two innovation approaches

Top down• Start from business functions perspective• Requires highly structured and mature IT-organization

Bottom up• Start from application integration perspective • Dutch Railways takes this approach

Page 37: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

37

Our bottom-up approach to innovation: ESB as the platform

1. FTP file transports over the ESB 2. Record (message) oriented data transport over the ESB – slicing into SOAP messages

and rebuild3. Shift focus from applications and batch-files to real-time point-to-point messages4. Scale up - start limited and let it grow 5. Define intermediate (canonical) layer of message types to decouple senders from

receivers

6. Decompose applications into documented – reusable - components (services)7. Start using tools for Business Process Management and Business Activity Monitoring

By connecting applications to the ESB, every application can have its own pace to innovationwithout affecting the other applications

Old world can talk with new world via the ESB

Incr

easi

ng m

atur

ity o

ver t

ime

A C DB= Transformation

Canonical

F FE

A C DB

E

Receiving applications

Sending applications

Step 3 Step 5

Page 38: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

38

SOA

SOA

SOA

SOA

Event-msg

Event-msgEvent-

msg

ERP

COTS

Legacy

Event-msg

ExternalGateway

ExternalSystems

Event-msg

Event-msg

SOA, COTS, ERP, legacy and external systems

= heterogenous and flexible application landscape with EDA =

Heterogenous systems are loosely (asynchronously) coupled via triggering event messages

Page 39: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

39

Competence Center of Integration

Page 40: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

40

Integration process

Competence Center of Integration to help

Define message flows (data analyses)

Build message flows through ESB

Buy/build/deploy adapters/wrappers

Buy/build/adapt/decompose business applications

Operate/manage message flows through ESB

Test and user acceptance

CCI

CCI advises development teams and delivers specialists (designers/developers)

CCI does the work

Focus is on message flows

Page 41: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

41

Deliverables of the CCI

Toward development teams:• Deliver (templates, solution patterns, guidelines)• Give advise (guidance during development projects)• Coordinate communciations between departments and with external parties• Enterprise wide metadatamanagement (XML-schema’s, semantics)• Deliver specialists (developers) to help with:

• Adapt legacy to connect to the ESB • Connect new applicaties/services to the ESB• Choose/build/configure adapters and wrappers • Configurate message flows (e.g. routing, security, enrichment, transformations)

• Facilitate test channels in the ESB• Facilitate separation of functional domains in the ESB• Functional adminstration of cross domain communication (interface-configuration)• If desired: functional administration of communication within domains • Functional life-cycle management of the ESB-infrastructure plus appurtenant tools• Deploy infrastructural ESB-facilities at distributed locations

- final slide -

Page 42: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

42

FIN

Page 43: Presentatie 20071121   Dutch Railways And Soa   Avans (1x90min)   V1.0

43

Questions?