business decisions from real-time processing of events

© 2008 TIBCO Software Inc. All Rights Reserved. Confidential and Proprietary.

1

Business Decisions from

Real-Time Processing of Events

Paul Vincent

CTO Business Rules and CEP

TIBCO Software inc.


2

Presenter

� Paul Vincent, CTO Business Rules and CEP, TIBCO Software

�Part of TIBCO Business Optimization product team

�Member OMG PRR and W3C RIF rules standards bodies

�Co-author CEP Blog http://tibcoblogs.com/cep

� TIBCO Software Inc.:

�Provides enterprise software that helps companies achieve service-

oriented architecture (SOA) and business process management

(BPM) success

• Event distribution via high performance messages / message buses

• Event processing for humans via BPM

• Event processing for real-time operations and analytics

� Complex Event Processing (CEP) product is TIBCO BusinessEvents

�Headquartered in Palo Alto, California

�Over 3,000 customers and offices in 40 countries


3

Our Problem


4

Ripped from the Headlines


5

Need to Know - in Real-Time

Must Quickly Discover and Manage Risks

Need IT Solutions to Support the Business, Faster

Fickle Customers – Little Brand Loyalty

Continual Competitive Pressure to Differentiate

Hard to adopt new tools within the old IT infrastructure

Business Need


6

But Business IT has its own Problems

Database-centric

Manual Processes and Slow Decisioning

Rigid IT Infrastructure

Need Better Technical Capability to Help Meet SLAs

Need Faster Time-to-Market


7

Big Problem for BothBusiness is Event-Driven - IT systems are Not

Need systems that provide answers,

Not make you ask questions *

* Anyway, most of the time we don’t know the right questions to ask

Risk


8


� Enable responsiveness and match the velocity of business

� Smart and aware solutions

that “learn” and “adjust” to circumstances

e.g. Traffic lights that sense congestion and re-evaluate timing policies as required


9

•Automating processes that previously could only be manually completed

•Opportunities can be identified and reacted to in a predictive way

•Automating processes that previously could only be manually completed

•Opportunities can be identified and reacted to in a predictive way




10

Example: Event-Driven Provisioning

Example: Event-Driven Provisioning

1.A delivery truck hits a deer and is unable to

complete the scheduled deliveries.

2.There are at least two active delivery trucks

ready in the same area within the last hour.

3.Solution: adjust, assign, then deliver to the

customer on time automatically without

manual processes.

1.A delivery truck hits a deer and is unable to

complete the scheduled deliveries.

2.There are at least two active delivery trucks

ready in the same area within the last hour.

3.Solution: adjust, assign, then deliver to the

customer on time automatically without

manual processes.


11

Common Issues

Event driven – but never just a “single event”

some sort of event correlation is needed

Time is of prime importance

before the opportunity ends

Decisions and Actions are required

driven by policies, practices, and procedures

decisions are based on the combination of

prior knowledge and present context


12

But…

My company is like this…

Which is where Complex Event Processing

(CEP) comes in.

But…

My company is like this…

Which is where Complex Event Processing

(CEP) comes in.


13

The Solution


14

1

Business Events form an “event cloud”

.


15

1

Applying intelligence to the Event Cloud

...

....

..

. .

.

... . .

..

.

.

..

..

. . .

..

..

. .

...

..

..


16

1

ID Theft Probability

Customer Upsell Scenario

Event detection

Finding Important Business Patterns

Real TimeAction / Reaction

Measuring GranularEvents

Complex Event Processing – What Is It?


17 17

Meeting Strategic GoalsCompany Insight Tailored to Executive NeedsPreparation for New Business Opportunities

Goals of CEP Solutions

Information Technology Management

C-Level

Line of Business Management

Security Policy GoalsSimplifying IT Landscape for “bigger picture”Service Assurance and Uptime

Meeting Business GoalsImprove Business Decision Making & ControlBetter Customer Service


18

Applications of CEP

� Adaptive Marketing – Sense & Respond

� Pattern: Capture opportunity with customer while ‘the window is open’.

� Telco – Situation Awareness

� SLA (Service Assurance)

� Real Time Service Offers and Analytics

� Finance – Sense & Respond

� Fraud Detection

� Track and Trace Trades/Deals/Settlements

� Pre/Post trade exceptions

� Logistics - Track & Trace

� Track Packages against a “Plan”. Infer package delays in a proactive manner. Alert customers.

� Government – Situation Awareness

� Track and Analyze ‘patterns’ that were otherwise very difficult to detect

� Dynamic Resource Scheduling – Sense and Respond

� Real Time Optimization of Resources against a “Plan”.


19

The Technology


20

Needed: the Event-Decision Architecture

Flexible SOA and Event-Driven Architecture


21

Event Processing, 1940s style

� Collossus - http://www.tnmoc.org/ColRbd.htm

� Searched for patterns in encrypted messages

� Hardware-based


22

Simple Event Processing: 1950s on

Data Source

Application

Database

Data Model

� Evolution of IT follows the simple event processing model: event at a time processing


23


Eve

nt

So

urc

es

Events and Messages

Eve

nt

Sin

ks

Application

Database

Data Model

Client ServerApplication

UI Model

Data Model



24


Eve

nt

So

urc

es

Event Bus

Eve

nt

Sin

ks

Application

Database

Data Model

BPM / Workflow

SOA Services

Client ServerApplication

UI Model

Data Model

Process Model


Incl. Decision Services

AnalyticServices

+ some refinements

Event Processing

- IF a Workflow / Service Event then process in SOA or BPM

- ELSE ignore or store in the DB for someone else to process

Real-World Event Processing

- Identify Business Events from Raw Events

- Then make Appropriate Decisions / Process / Services


25

� Onset of complex event processing models: continuous correlations of events

CEP Tool

Generalized CEP

Eve

nt

So

urc

es

Event Bus

Event Processing

Eve

nt

Sin

ks

Event Model Class Model Libraries

Local Event Store

High Performance CEP

- Event Processing scripts for high-performance model-to-code event-driven event-correlation applications

Low Latency CEP

- In-Memory event and data store for maximum performance


26

CEP Tool

Rule-driven Event Processing: real-time decisions

Eve

nt

So

urc

es

Event Bus

CEP Rule Elements

InferenceRule Engine

Rulebase

Eve

nt

Sin

ks

Event Model

Local Event Store

Intelligent CEP

- Inference Rule Engine for

declarative rules and high-

performance pattern-matching, via code generation for performance

Class Model Libraries


27

CEP Tool

State-driven Event Processing: decision lifecycles

Eve

nt

So

urc

es

Event Bus

CEP State Elements

State Engine

State Model

Eve

nt

Sin

ks

Event Model

Local Event Store

Dynamic State for CEP

- State Machine for event lifecycle modeling via states and state transition rules



28

CEP Tool

Query-based Event Processing: set-based decisions

Eve

nt

So

urc

es

Event Buse.g. TIBCO EMS, TIBCO RV

Eve

nt

Sin

ks

Event Model

Local Event Store

Event Stream Processing

- Continuous queries for

dynamic patterns / facts across event streams, using

SQL-type constructs

Query Elements

Query Engine

Query List



29

CEP Tool

Distributed Event Processing: resilient scalable decisions

Eve

nt

So

urc

es

Event Bus

CEP Agent

State Engine


State Model

Rulebase

Eve

nt

Sin

ks

Event Model

Shared Event Store

Scalable CEP

- Replicated CEP Agents for

load-balancing and hot-standby


CEP Agent

State Engine


State Model

Rulebase


30

Distributed CEP Framework

Managed Event Processing: controlled decisions

Eve

nt

So

urc

es

Event Bus

CEP Rule Agent

State Engine


State Model

Rulebase

Data Grid

Eve

nt

Sin

ks

CEP Query Agent

Query Engine

Query List

DecisionManagement

CEP Editors Managed Entities


RealtimeDashboard

AJAX

CEP Framework

- Event Processing Elementmultiple types (rule, state, query, …)

for flexible event processing

Managed CEP

- workflow-enabled

Decision Managementfor Business control of CEP Elements

CEP Platform

- distributed Event Processing Agentsgives scalability in volume + complexity

Shared Low-Latency Data

- distributed event history Data Grid

for Low Latency High

Volumes information


31

Distributed CEP Framework

Summary of CEP’s Benefits for Real-time Decisions

Eve

nt

So

urc

es

Event Bus

CEP Rule Agent

State Engine


State Model

Rulebase

Data Grid

Eve

nt

Sin

ks

CEP Query Agent

Query Engine

Query List

DecisionManagement

CEP Editors Managed Entities


RealtimeDashboard

AJAX

Champion Challenger thru multiple ruleset control / multiple decision agents

Business Events drive business rule execution

Rule maintenance & execution using BRE, BRMS features

Real-time analytics using real-time rule monitoring + statistical functions / agents driving rule & score changes

on-the-fly

Operational event store

provides event warehouse for real-time historic pattern detection

IT-friendly model-driven

engineering via easy-to-understand state, query, rules, concepts, …


32

Leading to: Operational Intelligence

Event Sources Event Bus

Event Sinks

Real-

tim

eS

tati

sti

cs

an

d A

naly

tics

Sta

tisti

cs

Bu

sin

ess

Co

ntr

ol

of

Even

t P

rocessin

g

Busin

ess

Editors

Man

ag

ed

En

titi

es

Even

tM

on

ito

rin

g

Gra

ph

ical

Dash

board

Concept Model

Event Model

Even

t-d

riven

Ru

les

Ru

leb

as

e

Sta

te

Tra

nsit

ion

sS

tate

Mo

del

Co

nti

nu

ou

sQ

ueri

es

Qu

eri

es

Oth

er

/ 3

rdP

art

yA

lgo

rith

ms

Neu

ral N

ets

, C

P, etc

Off

lin

e B

usin

ess

Sta

tisti

cs

an

d A

naly

tics

Vie

ws

Vis

ual

Rep

ort

s

Distributed Data Grid

Complex Event Processing Agents

Complex Event Processing Platform.

Operations Intelligence Infrastructure

Complex Event Processing Network

*


33

We Interrupt This Presentation …

� Syndera Corporation

� A few weeks ago TIBCO acquired certain assets of Syndera Corporation, a provider of real-time business intelligence software.


34

The Proof


35


36

CEP Users

� Financial Services (banking, investments) are just one market for CEP technology

� Other users in the “real time” space include

�Transport / Logistics

�Telco

�Manufacturing

�Military / Intelligence

EDA White Paper- downloads by industry

courtesy Brenda Michelson, Elemental Links

@ 4th EPTS mtg, Sep08

http://elementallinks.typepad.com/bmichelson/2006/02/eventdriven_arc.html


37

Situation Awareness: Airline Operations

Problem: Airline Operations staff need to get early alerts of possible operational problems

Event sources: aircraft, dispatch, weather, replenishers, passenger information system, …

Complex events: aircraft delays, reroutes, breakdowns…

Output: dashboard for operations

Project context: 1st CEP project, live from Aug08

Technology: TIBCO EMS backbone, + TIBCO BW for SOA orchestration+ TIBCO GI for AJAX-based dashboard+ TIBCO BE for CEP


38

Sense & Respond: Commercial Banking

Problem: Accuracy, timeliness for customer marketing; justify SOA re-engineering

Event sources: WWW, CRM, ATM, financial TXs, …

Complex events: customer life events, …

Output: scripts + strategies for CRM operators

Project context: 1st CEP project to touch all customers in a world region, live from late 07

Technology: TIBCO EMS + IBM MQ backbones, + TIBCO AM and AM BW for SOA + TIBCO iProcess for BPM+ TIBCO GI AJAX + IBM portal for UIs+ TIBCO BE for CEP

Computer says….

No…


39

Track n Trace: System Monitoring in Insurance

Problem: Monitor / track / report on system events and processes regarding policy documents

Event sources: BPM, …

Complex events: process exceptions, …

Output: dashboards, BI reports, ETL controls

Project context: 1st CEP project, live from mid 08

Technology: TIBCO EMS backbone, + 3rd party BPM+ 3rd party UIs+ TIBCO BW for SOA+ TIBCO BE for CEP


40

2009 Predictions

� CEP continues to push the boundaries for IT

� Distributed Decision Engines

� Event-enabled Decision Management Workflow

� CEP continues to gain more acceptance for real-time STP

� CEP tools provide the new “best practices” –performance, decisions, persistence, reliability,

model-driven, XTP, built-in-BAM, standards …

� CEP growing 20-50% YOY

� Market consolidation continues

� 2008 CEP acquisitions by IBM, Oracle

� Compare the pure-play BRE acquisitions


41

CEP case studies at theTIBCO User Group 2008

included:

Carphone Warehouse (UK)AllState (US)

Citi (Asia)SWA (US)

alsoTIBCO Service Performance Manager

TIBCO Spotfire for SixSigma

More real-time decisioning

case studies will be at:

http://tucon.tibco.com/ THE END

For CEP and rule technologyupdates see the

TIBCO Complex Event Processing Bloghttp://tibcoblogs.com/cep


42

END

business decisions from real-time processing of events

Documents