operational intelligence with wso2 bam
DESCRIPTION
TRANSCRIPT
![Page 1: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/1.jpg)
Gaining Opera+onal Intelligence �with WSO2
BAM
Director, Research, WSO2 Inc. Visi6ng Faculty, University of Moratuwa Member, Apache SoEware Founda6on
Research Scien6st, Lanka SoEware Founda6on
Srinath Perera
![Page 2: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/2.jpg)
About WSO2 ๏ Global enterprise, founded in 2005
by acknowledged leaders in XML, web services technologies, standards and open source
๏ Provides only open source plaHorm-‐as-‐a-‐service for private, public and hybrid cloud deployments
๏ All WSO2 products are 100% open source and released under the Apache License Version 2.0.
๏ Is an Ac+ve Member of OASIS, Cloud Security Alliance, OSGi Alliance, AMQP Working Group, OpenID Founda+on and W3C.
2
๏ Driven by Innova+on
๏ Launched first open source API Management solu+on in 2012
๏ Launched App Factory in 2Q 2013
๏ Launched Enterprise Store and first open source Mobile solu+on in 4Q 2013
![Page 3: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/3.jpg)
3
What WSO2 delivers
![Page 4: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/4.jpg)
Business Model
4
![Page 5: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/5.jpg)
More Informa+on !
๏ Include links to product downloads, white paper downloads , etc.
5
![Page 6: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/6.jpg)
Outline § Connected Business and Big data analy+cs § Big Data Technologies from WSO2
§ BAM – Batch analy+cs § CEP – Real +me analy+cs § Lambda Architecture to combine
§ From your business to insights § Iden+fy KPIs § Calculate KPIs § Build Dashboards § Add Drilldown § Alerts and Ac+ons
![Page 7: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/7.jpg)
About Connected Business webinar series
§ March 05:Introduction to the Connected Business
§ March 12: Enterprise Integration made easy with WSO2 ESB
§ .. § April 30: Gaining
Operational Intelligence with WSO2 BAM
§ May 7: The WSO2 Advantage for a Connected Business
![Page 8: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/8.jpg)
Adap+ve Connected Business
![Page 9: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/9.jpg)
Connected Business
![Page 10: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/10.jpg)
Be Adap+ve ๏ Capture business ac+vity (iden+fied by messages, transac+on execu+on, and data state changes) and store data points for future analy+cs
๏ Deliver automated no+fica+ons to stakeholders and systems based on business ac+vity, stakeholder accountability, and authority.
๏ Automa+cally adapt business process execu+on based on events and current condi+ons
![Page 11: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/11.jpg)
Big Picture
![Page 12: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/12.jpg)
Collec+ng Data ๏ Data collected at sensors and sent to big data system via events or flat files
๏ Event Streams: we name the events by its content/ originator
• Get data through – Point to Point – Event Bus
• E.g. Data bridge – a thrib based transport we did that do about 400k events/ sec
![Page 13: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/13.jpg)
Making Sense of Data ๏ To know (what happened?)
๏ Basic analy+cs + visualiza+ons (min, max, average, histogram, distribu+ons … )
๏ Interac+ve drill down
๏ To explain (why)
๏ Data mining, classifica+ons, building models, clustering
๏ To forecast
๏ Neural networks, decision models
![Page 14: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/14.jpg)
Dashboards and last Mile § Presenting information
o To end user o To decision takers o To scientist
§ Interactive exploration § Sending alerts
http://www.flickr.com/photos/stevefaeembra/3604686097/
![Page 15: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/15.jpg)
![Page 16: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/16.jpg)
Big Data Architecture
![Page 17: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/17.jpg)
Data Collec+on
• Can receive events via SOAP, HTTP, JMS, ..
• WSO2 Events is highly op+mized version (400K events TPS)
• Default Agents and you can write custom
agents.
Agent agent = new Agent(agentConfiguration); publisher = new AsyncDataPublisher(
"tcp://localhost:7612", .. ); StreamDefinition definition =
new StreamDefinition(STREAM_NAME, VERSION);
definition.addPayloadData("sid", STRING); ... publisher.addStreamDefinition(definition); ... Event event = new Event(); event.setPayloadData(eventData); publisher.publish(STREAM_NAME, VERSION, event);
![Page 18: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/18.jpg)
Business Ac+vity Monitor
![Page 19: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/19.jpg)
BAM Hive Query Find how much +me spent in each cell.
CREATE EXTERNAL TABLE IF NOT EXISTS PlayStream …
select sid,
ceiling((y+33000)*7/10000 + x/10000) as cell, count(sid)
from PlayStream
GROUP BY sid, ceiling((y+33000)*7/10000 + x/10000);
![Page 20: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/20.jpg)
Complex Event Processor
![Page 21: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/21.jpg)
CEP Query define partition sidPrt by PlayStream.sid, LocBySecStream.sid
from PlayStream#window.timeBatch(1sec)
select sid, avg(x) as xMean, avg(y) as yMean, avg(z) as zMean
insert into LocBySecStream partition by sidPrt
from every e1 = LocBySecStream ->
e2 = LocBySecStream [e1.yMean + 10000 > yMean
or yMean + 10000 > e1.yMean]
within 2sec select e1.sid
insert into LongAdvStream partition by sidPrt ;
Calculate the mean location of each player
every second
Detect more than 10m run
![Page 22: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/22.jpg)
Lambda Architecture
![Page 23: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/23.jpg)
![Page 24: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/24.jpg)
Extrac+ng Opera+onal Intelligence from your
Business
![Page 25: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/25.jpg)
Key Performance Indicators (KPIs) ๏ To monitor, you need to measure
๏ KPIs are the bojom line
๏ For a organiza+on it may be profit, or revenue
๏ For marke+ng, it may be LEADs generated
๏ They are oben domain specifics
๏ For each KPI, we need to see history, find rela+onships, and some+me possible future trends.
![Page 26: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/26.jpg)
Many Stake Holders, Many Views
![Page 27: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/27.jpg)
Decision Process
![Page 28: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/28.jpg)
Dashboard
๏ Idea is to get the “Overall idea” in a glance
๏ Like your car dashboard
๏ Support for personaliza+on, you can build your own dashboard.
๏ This is the entry point for Drill down and ac+vity monitoring.
![Page 29: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/29.jpg)
Drill Down
๏ Find the problem, and find what caused it.
๏ It is like “Finding CPU bojleneck”
๏ Idea is to recursively find categories which are responsible for the problem
๏ E.g. See year’s sales, then drill down to a par+cular year to see per month data.
๏ See country level data, and drill down to see state wide data.
๏ Ac+vity monitoring is a key ac+vity here
![Page 30: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/30.jpg)
Case Study: Sobware Support Services
๏ KPIs
๏ Number of open Issues
๏ Mean and Max +me to close a issue
๏ SLA viola+ons
๏ List of long running issues
๏ Issue by last update +me
๏ Data
๏ Collect data when issue is created or updated or closed.
๏ E.g. IssueChange(id, ac+on, +me, userID)
๏ Emit ID as the correla+onID
![Page 31: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/31.jpg)
From data to KPI ๏ Group issues by ID grouping each issue into one ac+vity (Ac+vity Monitoring)
๏ Calculate KPIs for each ac+vity
define partition IssueParition IssueChange.id;
from IssueChange[type=“open”] as s
->IssueChange[“closed”] as e
select e.time-s.time ..
Aggregate data in different level to support drill down
๏ hour, weekly, and monthly
๏ Ver+cal, product,
![Page 32: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/32.jpg)
Aggregate to support Drilldown
Dimension 1 Dimension 2
![Page 33: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/33.jpg)
Dashboard (Build with UES)
Click to drill down
Click to drill down
Click to see details about Issues
![Page 34: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/34.jpg)
Drill Down
๏ Click and drill down issues in each category
๏ Can select a issue and see all ac+vi+es happened within the issue
๏ See who takes most +me on issues
๏ Look at day, week, month trends
![Page 35: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/35.jpg)
Analy+cs Process
![Page 36: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/36.jpg)
Ques+ons?
![Page 37: Operational Intelligence with WSO2 BAM](https://reader034.vdocument.in/reader034/viewer/2022052310/54c6a92d4a7959aa4b8b4573/html5/thumbnails/37.jpg)
THANK YOU!!!
37