© 2008 mindtree consulting© 2010 mindtree limited creating an event-driven soa madhusudhan km...
TRANSCRIPT
© 2008 MindTree Consulting© 2010 MindTree Limited
Creating an Event-Driven SOA
Madhusudhan KMChief Architect & Head – Central Architecture Group, IT Services, MindTree Ltd
© 2010 MindTree Limited
Topics
Concepts, Sample Use case & EDA characteristics
EDA Reference Architectures
Present & Future applications of EDA
Major players in EDA
Summary & Conclusion
Slide 2
© 2010 MindTree Limited
Event driven world
We all live in an event driven world
All animals along with human beings respond to events of two types ; opportunities & threats
Zebras on the savannah sense and respond to the opportunity of a water hole or the threat of a lion.
Banks track credit card usage to stop fraudulent charges as they occur
We pick telephone calls when it rings. We do not pick other’s telephone (usually)…sometimes, we also filter our own calls if the number if unknown
We have cognitive sensors and filters that help us to concentrate on events that are important to us and ignore events that are not related to us (or at least we think they are not related !!)
Slide 3
Are enterprises using sensors to respond to business opportunities and to fight threats at right
time ??
© 2010 MindTree Limited Slide 4
Events – Airline Use Case
Flight gets
delayed
• I will miss my connecting flight• I will have to cancel my hotel
reservation and car rental booking
• I will have to cancel my meetings
• We can’t serve the next flight to start from destination of this flight
• Ground operations staff at destination will have to arrange alternate crew
• Make sure the passengers are accommodated on the next connecting flight
• Book hotels for passengers if the wait time is more than six hours
BLR-Dubai-LHR
Dubai AirportCabin crew to serve another flight in Dubai
© 2010 MindTree Limited
Event - Flight delayed
Impact on passenger traveling for business purposes
Might miss a connecting flight
Might need to cancel/change hotel reservation & car rental reservation
Might need to cancel/postpone a meeting
Impact on Airline
The flight crew may miss their next flights. The flights must be delayed or replacement crews assigned.
Crew members who are passengers on the current flight may not be available for their assigned flights.
Catering provider might have to be notified of this event if the meal preparation should be delayed / cancelled.
If there are priority passengers on the flight, airlines may want to see how these priority passengers are affected and what could the airline do to reduce the inconvenience to priority passengers.
Slide 5
Events – Airline Use Case
© 2010 MindTree Limited
What is an Event ?
An event is a notable thing that happens inside or outside your business. An event (business or system) may signify a problem or impending problem, an opportunity, a threshold, or a deviation.
Examples
A financial trade
An airplane lands
A sensor outputs a reading
A change of state in a database or a finite state machine
A key stroke
A natural occurrence such as an earthquake
Slide 6
Events – Concepts & Terms
© 2010 MindTree Limited
Simple Event : Any event in its simplest form
Flight delay event
Complex Event : An event that is an abstraction of other events called its members.
The 1929 stock market crash – an abstraction denoting many thousands of member events, including individual stock trades)
The 2004 Indonesian Tsunami – an abstraction of many natural events
Derived event : An event that is generated as a result of applying a method or process to one or more other events.
An event reporting that company B has entered the bidding to take over A with probability 0.9, might be derived from an event reporting that the price of company A’s stock has jumped 10% in 5 minutes.
The absence of an event, say in a given time interval, can lead to a derived event reporting that the first event did not
Slide 7
Events – Concepts & Terms
© 2010 MindTree Limited
Event Driven Architecture : An architectural style in which some of the components are event driven and communicate by means of events.
Simple Event Processing: In simple event processing, a notable event happens, initiating downstream action(s). Simple event processing is commonly used to drive the real-time flow of work—taking lag time and cost out of a business.
Stream Event Processing : In stream event processing, linearly ordered stream of events are processed
Events (orders, RFID transmissions, etc) are both screened for notability and streamed to information subscribers.
Stream event processing is commonly used to drive the real-time flow of information in and around the enterprise––enabling in time decision making
A stream may contain events of different types
Slide 8
Events – Concepts & Terms
© 2010 MindTree Limited
Complex Event Processing : A sophisticated form of EDA that is used to extract the information value from multiple events.
CEP systems find patterns in event data to detect opportunities and threats. Timely alerts are then pushed to the appropriate recipients, often using Business Activity Monitoring (BAM) dashboards. The result is faster and better operational decisions and more timely responses.
In CEP, events usually across types, occur over long period of time
Event correlation can be casual / temporal / spatial
Few possible of application of CEP are Algorithmic trading, Fraud monitoring and detection, Global epidemic warning, Air traffic control, Self tuning databases, Adaptive middleware, etc.
Slide 9
Events – Concepts & Terms
© 2010 MindTree Limited Slide 10
Event Driven Architecture – Few characteristics
Model SOA EDA
Interaction Request – ResponseRequest – Callback
Sense - RespondPub – Sub
Invocation Client/user driven Event driven
Relationship 1 : 1Client : Service provider
1 : nEvent source : Event sinks
Flow Top downClient Service provider data
Complex flow
© 2010 MindTree Limited Slide 11
Events – Airline Use Case
Event Producer
Event Management
Engine
Event Consumer
Event Consumer
Event Consumer
SMS to Passenger – Your Flight delayed ; You are booked on alternate flight
Trigger a business process in Airline reservations to take care of rebooking passengers who missed connecting flights
Trigger a business process in Flight operations to arrange new set of cabin crew for the flight to start at destination
© 2010 MindTree Limited Slide 12
Events – Tying concepts together
Event Hub
Event Producers
Event Consumers
Consumers receive events based on rules, filters, etc.
Producers could be business processes, services/transactions, state change in state of entity in DB, user actions on applications, etc
Consumers have no obligation to producers
Consumers are usually producers too
© 2010 MindTree Limited Slide 13
Events – Moving from Loose coupling to Minimal coupling
Event
Event
EventProcess Service Service
Legacy
Data Store
Service
Business Processes and Services are not hardwired…..instead, they are linked via events
© 2008 MindTree Consulting© 2010 MindTree Limited
Event Driven SOA – Reference Architectures
© 2010 MindTree Limited Slide 15
A framework for handling highly concurrent systems
© 2010 MindTree Limited Slide 16
A framework for handling highly concurrent systems
© 2010 MindTree Limited Slide 17
A framework for handling highly concurrent systems
© 2010 MindTree Limited Slide 18
Staged Event Driven Architecture - SEDA
Each Stage contains an incoming event queue, a thread pool, an event handler and a controller
Each stage can be independently managed
Stages can be run in series or in parallel or a combination of the two
Each stage is separated by a queue
Use of event queues allows each stage to be individually load conditioned by using proper thresholds
© 2010 MindTree Limited Slide 19
Benchmark of HTTP servers
© 2010 MindTree Limited Slide 20
Simple Event Processing
Source : Event-Driven Architecture Overview by Brenda M. Michelson, Sr. VP and Sr. Consultant, Patricia Seybold Group
© 2010 MindTree Limited Slide 21
Stream Event Processing
Source : Event-Driven Architecture Overview by Brenda M. Michelson, Sr. VP and Sr. Consultant, Patricia Seybold Group
© 2010 MindTree Limited Slide 22
Complex Event Processing
Source : Event-Driven Architecture Overview by Brenda M. Michelson, Sr. VP and Sr. Consultant, Patricia Seybold Group
© 2010 MindTree Limited
Analyst driven BI
Ad-hoc queries, Periodic reports, data mining, etc
Output is mainly used for planning
Process driven BI
Some process driven BI systems use near real time information using BAM (Business Activity Monitoring)
Input for BAM is event data that arrived in the last few seconds/minutes
Historical data is used to put the new data in in context and to enrich the information before it is distributed
Strategy driven BI
Used to measure and manage overall business performance
Are we on track to meet our monthly target ?
Slide 23
Event Processing in Business Intelligence
© 2010 MindTree Limited Slide 24
Periodic intelligence v/s Continuous intelligence
Source : Event Processing : Designing IT systems for Agile companies by Mani Chandy & Roy Schutle
Event driven system sends alert when things occur and not based on predetermined schedule or upon receiving ad-hoc query
Reports complex events like exceptions events on KPI
They operate fast enough to allow operations staff to intervene in a transaction when it is still in flight
© 2008 MindTree Consulting© 2010 MindTree Limited
Few applications of EDA & CEP
© 2010 MindTree Limited
Algorithmic trading
Algorithmic trading is achieved using computer algorithms deciding on aspects of the order such as timing, price & quantity of the order
CEP engines are now used to predict a trend to make use of both sides of the market (ups & downs)
As of 2006, a third of all EU and US stock trades were driven by automatic programs or algorithms
Multiple players in capital market segment are using CEP engines for algorithmic trading
Bluecrest, Wall street on demand (WSOD), SunGard, etc
Slide 26
© 2010 MindTree Limited
Fraud detection
A 2006 CyberSource report stated that 2.8 billion USD was lost to online fraud in US & Canada in 2005
Requirements
Stop new account setups from fraudulent IP addresses
Verify user identity in every user transaction based on click behavior
Identify multiple users trying to login from same IP address
Identify single user logins from multiple IP addresses within a time span
Fraud detection systems enabled with CEP products are being used in Banks, Betting companies, Insurance companies, etc
In Banks, multiple withdrawals made from a card within a specific time interval from locations that are distance apart indicates a fraud
Slide 27
© 2010 MindTree Limited
Air Traffic Control
Autonomous event-driven air traffic management system spanning the world.
Why is it needed ?Crowded air space
The technology used in ATC is dated (1960)
Slide 28
Source : The Future Event Driven World Scenario 1: Air Traffic Management by David Luckham
© 2010 MindTree Limited
Business Intelligence
• Algorithmic Trading
• Fraud Detection• Event-Driven SOA• Business Activity
Monitoring• Supply Chain
Visibility
Autonomic Computing
• Self-tuning databases
• Planning systems• Self configuring
hardware and software systems
• Adaptive middleware
Travel Industry
• Air Traffic Control• Baggage Tracking
and Tracing• Passenger
Security• Automatic re-
accommodation• Intelligent
baggage tracking• Aircraft
maintenance based on real time information
Potential future applications
Slide 29
© 2010 MindTree Limited
Health Care
• Global Epidemic Warning
• Hospital Patient Monitoring
• Home Monitoring Systems
• RFID In-Body Monitoring
Ambient Intelligence
• Smart Homes• Smart Cities• Personalized
Mobile Information Systems
Homeland Security
• Storm Evacuation Systems
• First Responder Systems
• Terrorism Situational Awareness
• Power, Fuel, and Water System Monitoring
Potential future applications
Slide 30
© 2010 MindTree Limited
Environmental Systems
• Wind Engineering• Weather Systems• Water Sources• Habitat
Conservation
Military Systems
• Command and control systems
• Physical Infrastructure Monitoring
Transportation Systems
• Traffic Management
• Produce Tracking Systems
• Truck Routing Systems
Potential future applications
Slide 31
© 2008 MindTree Consulting© 2010 MindTree Limited
Major Players in EDA & CEP space
© 2010 MindTree Limited
Event Pattern Matching
Event Stream Processing
Ability to support multiple event patterns
Filter, Enrich, Split, Aggregate, Correlate
SQL like Event Query Language support
Variety of event data storage options
Support for multiple event type representations like Java Object & XML
Multiple integration options
Tools for event modeling & visualization with rich IDE along with debugging features
Event history replay
JMX instrumentation
Ability to expose services and to integrate into ESB productsSlide 33
What to look for in EDA & CEP Products
© 2010 MindTree Limited Slide 34
EDA & CEP – Major Players
BusinessEvents
Oracle CEP ; Able to process 1 million events/sec with low latency
Apama Event Processing Platform ; Specialize in solutions to capital markets like Algorithmic trading, Smart order routing, etc Aleri Streaming Platform; Specialize in stream event processing
Event Processing Platform ; Specialize in stream event processing
Event Processing & CEP Platform ; Open source CEP platform
Vhayu Velocity : High performance solution used widely for tick data management allowing users to perform real-time and historical market data analysis in a single CEP platform.
Websphere Business Events
© 2008 MindTree Consulting© 2010 MindTree Limited
Summary & Conclusion
© 2010 MindTree Limited
Organizations should gear up to sense opportunity and threat events from both inside and outside
Event Driven Architecture combined with SOA will enable organizations to achieve this
Technology products to support EDA+SOA are maturing
There are many interesting futuristic use cases that are possible in different domains
Slide 36
Summary & Conclusion
So…Let us sense this opportunity and respond by creating some interesting solutions
© 2010 MindTree Limited
Acknowledgements
Understanding Event-Driven Architecture : Article by Roy Schutle and Dr K Mani Chandy
Event Processing Glossary by David Luckham, Roy Schulte
The Future Event Driven World Scenario 1: Air Traffic Management by David Luckham
A Design Framework for Highly Concurrent Systems by Matt Welsh, Steven D. Gribble, Eric A. Brewer, and David Culler, Computer Science Division University of California, Berkeley
SEDA: An Architecture for Well conditioned, Scalable Internet Services by Matt Welsh, David Culler, and Eric Brewer Computer Science Division University of California, Berkeley
Event-Driven Architecture Overview, Event-Driven SOA Is Just Part of the EDA Story by Brenda M. Michelson Sr. VP and Sr. Consultant, Patricia Seybold Group
Slide 37
© 2010 MindTree Limited Slide 38 © 2010 MindTree Limited
Madhusudhan KMChief Architect & Head – Central Architecture Group, IT Services,
MindTree [email protected]
+91- 99860 48076www.mindtree.com
Successful
Customers
Happy People
Innovative
Solutions
Successful
Customers
Happy People
Innovative
Solutions
Our Mission