real-time analytics: from data to actions in milliseconds

49

Upload: wso2

Post on 11-Aug-2015

326 views

Category:

Technology


4 download

TRANSCRIPT

●●●●●●●●

’s omplex Event Processing?

●●●●

’s omplex Event Processing?

●●●●

CEP

Event Flow of WSO2 CEP

●●●●●

public class Event { String streamId; long timestamp; Object[] data;}

●○○

●●●

■●●

■■

define stream <event stream>(<attribute> <type>,<attribute> <type>, ...);

from <event stream>select <attribute>,<attribute>, ...insert into <event stream> ;

define stream SoftDrinkSales (region string, brand string, quantity int,

price double);

from SoftDrinkSalesselect brand, quantityinsert into OutputStream ;

define stream OutputStream(brand string, quantity int);

Output Streams are inferred

define stream SoftDrinkSales (region string, brand string, quantity int,

price double);

from SoftDrinkSalesselect brand, avg(price*quantity) as avgCost,‘USD’ as currencyinsert into AvgCostStream

from AvgCostStreamselect brand, toEuro(avgCost) as avgCost,‘EURO’ as currencyinsert into OutputStream ;

Enriching Streams

Using Functions

define stream SoftDrinkSales (region string, brand string, quantity int,

price double);

from SoftDrinkSales[region == ‘USA’ and quantity > 99]select brand, price, quantityinsert into WholeSales ;

from SoftDrinkSales#window.time(1 hour)select region, brand, avg(quantity) as avgQuantitygroup by region, brandinsert into LastHourSales ;

Filtering

Aggregation over 1 hour

Other supported window types: timeBatch(), length(), lengthBatch(), etc.

define stream Purchase (price double, cardNo long,place string);

from every (a1 = Purchase[price < 10] ) -> a2 = Purchase[ price >10000 and a1.cardNo == a2.cardNo ]

within 1 dayselect a1.cardNo as cardNo, a2.price as price, a2.place as placeinsert into PotentialFraud ;

define stream StockStream(symbol string, price double, volume int);

partition by (symbol of StockStream)begin from t1=StockStream, t2=StockStream [(t2[last] is null and t1.price < price) or

(t2[last].price < price)]+within 5 min

select t1.price as initialPrice, t2[last].price as finalPrice, t1.symbol

insert into IncreaingMyStockPriceStream end;

define table CardUserTable (name string, cardNum long) ;

@from(eventtable = 'rdbms' , datasource.name = ‘CardDataSource’ , table.name = ‘UserTable’, caching.algorithm’=‘LRU’)define table CardUserTable (name string, cardNum long)

●●

define stream Purchase (price double, cardNo long, place string); define stream CardUserStream (name string, cardNo long) ;

define table CardUserTable (name string, cardNum long) ;

from Purchase#window.length(1) join CardUserTableon Purchase.cardNo == CardUserTable.cardNum

select Purchase.cardNo as cardNo, CardUserTable.name as name, Purchase.price as price

insert into PurchaseUserStream ;

from CardUserStreamselect name, cardNo as cardNumupdate CardUserTable

on CardUserTable.name == name ;

Similarly insert into and delete are also supported!

●●●●

define stream SalesStream(brand string, price double, currency string);

from SalesStreamselect brand, custom:toUSD(price, currency) as priceInUSDinsert into OutputStream ;

Reffer with namespace

●●●

●●●●●●

●●

●●●

With configurable alerting & Monitoring capabilities.

Why not use Apache Storm ?

WSO2 CEP += Apache Storm

How we scale ?

Scaling with Storm

Siddhi QL

define stream StockStream (symbol string, volume int, price double);

@name(‘Filter Query’)from StockStream[price > 75]select *insert into HighPriceStockStream ;

@name(‘Window Query’)from HighPriceStockStream#window.time(10 min)select symbol, sum(volume) as sumVolume insert into ResultStockStream ;

Siddhi QL - with partition

define stream StockStream (symbol string, volume int, price double);

@name(‘Filter Query’)from StockStream[price > 75]select *insert into HighPriceStockStream ;

@name(‘Window Query’)partition with (symbol of HighPriceStockStream)begin

from HighPriceStockStream#window.time(10 min)select symbol, sum(volume) as sumVolume insert into ResultStockStream ;

end;

Siddhi QL - distributed

define stream StockStream (symbol string, volume int, price double);

@name(Filter Query’)@dist(parallel= ‘3')from StockStream[price > 75]select *insert into HightPriceStockStream ;

@name(‘Window Query’)@dist(parallel= ‘2')partition with (symbol of HighPriceStockStream)begin

from HighPriceStockStream#window.time(10 min)select symbol, sum(volume) as sumVolume insert into ResultStockStream ;

end;

On Storm UI