![Page 1: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/1.jpg)
DISRUPTION OF THE APM MARKET THROUGH
OPEN STANDARDS AND OPEN SOURCE -
OUR VISION OF AN IDEOLOGICAL CHANGE
09. MAI 2017
CHRISTIAN VÖGELE, STEFAN THIEME, ALEXANDER WERT
SOFTWARE PERFORMANCE MEETUP
![Page 2: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/2.jpg)
History of APM Tools: What happened so far?
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 2
http://4.bp.blogspot.com/-uEdWYuTHUnU/VMSClSFgTcI/AAAAAAAAAPw/uV2YelL9luc/s1600/3-tier.PNG
https://www.manageengine.com/network-monitoring/images/cpu_memory_disk_monitoring.gif http://i176.photobucket.com/albums/w163/GodDreamsOfVegas/nerd-46422.jpg
IT Department
![Page 3: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/3.jpg)
Status quo: Situation has changed
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 3
Multi-tier architecture Users throughout organization Tool diversification
Data consumers
Development Operations Business
![Page 4: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/4.jpg)
Multi-Vendor, Multi-Tool
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 4
Development Operations Business
Tools Tools Tools
APM
Data exchange (cross-tool)
Data aggregation
Big data analytics
Business-specific APM
![Page 5: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/5.jpg)
Licence Costs
Vendor Lock-In
Typical Model: x $ per Agent Scaling?
Weakness of commercial APM tools
Microservices Internet of Things
[Abbildung: http://blog.wso2.com] [Abbildung: Christian Hinkelmann, http://nahverkehrhamburg.de]
Flexibility Interoperability Sustainability
Mobile Revolution
[Abbildung: https://uxmag.com]
Adaption to requirements
of company
Error Correction
Data Sources
Analysis Tools
Other APM Tools
Product is not developed
anymore
Strategy Change of APM Vendor
![Page 6: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/6.jpg)
Disruption in the APM tool market
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 6
Agent - Tool A
Agent - Tool B
Tool A
Tool B
Tool X
Agent - Tool A
Agent - Tool B
Tool A
Tool B
Tool X
* data mapping + conversion
*
*
* * *
Universal Agent
Universal Agent
Tool A
Tool B
Tool X
data mapping
+ conversion
Universal Agent
Universal Agent
Tool A
Tool B
Tool X
Unified specification
![Page 7: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/7.jpg)
What if?
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 7
Shift focus from collection to usage
Usage
Data Collection
Data Collection
Data Collection
Today
• Commercial tools provide proprietary data
• Data of different tools hard to exchange
• Data of different tools hard to be aggregated
• Commercial tools hardly integrate with free tools
• Company-wide business monitoring not possible
• Current setups support APM maturity only in parts
• Licensing based on low-level (CPU cores, #JVMs)
• …
Future
• Commercial and free tools use open data formats
• Data exchange allows triage big picture analysis
• Data aggregation allows (big data) analytics
• Commercial tools integrate with free tools
• Aggregated data enables business monitoring
• APM maturity process adoption within enterprises
• Licensing based on medium-level KPIs (i.e. usage)
• …
![Page 8: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/8.jpg)
It‘s about Collaboration
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 8
Performance
Modeling
Web Performance
Analysis
System &
Resources Monitoring Real User Monitoring
JRat
JMemProf
Low-Level
Performance Profiling
Monitoring &
Application Deep Dive
![Page 9: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/9.jpg)
Let‘s get concrete .. OPEN.xtrace
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 9
OPEN.xtrace format enables
• data interoperability and exchange between APM tools
• SPE researchers to develop their approaches in a tool-agnostic and comparable manner
• Modelling with the information that is available in leading APM tools today
OPEN.xtrace is a community effort as part of the overall goal to increase interoperability of SPE/APM
techniques and tools.
Agent - Tool A
Agent - Tool B
APM Tool
* Trace adapter
*
*
OPEN.xtrace
![Page 10: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/10.jpg)
Let‘s get concrete .. Distributed Tracing
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 10
A basic distributed trace:
• Distributed tracing has the challenge to monitor transactions propagating across a distributed system
• Distributed tracing is a difficult task as instrumentation must propagate the tracing context both within
and between processes:
• A lot of engineer effort
• Monkey-patched communication packages
• Inconsistencies across platforms
• Not reasonable to use a single tracing vendor
• Traces are truncated in case no mechanism for trace description and propagation is used
http://opentracing.io/documentation/, https://medium.com/opentracing/towards-turnkey-distributed-tracing-5f4297d1736
A trace is a directed
acyclic graph (DAG) of
"spans": named, timed
operations representing
a contiguous segment of
work in that trace.
![Page 11: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/11.jpg)
Let‘s get concrete .. Distributed Tracing
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 11
OpenTracing is a open distributed tracing standard
• What to standardize? (Encoding formats and/or standard instrumentation APIs)
• OpenTracing allows developers of application code, OSS packages, and OSS services to instrument
their own code without binding to any particular tracing vendor
• OpenTracing provides API standards and allows the concrete implementations to take control over
encoding formats; this in turn gives the application owner the flexibility to switch (or add) tracing vendor.
https://medium.com/opentracing/towards-turnkey-distributed-tracing-5f4297d1736 OSS: Operations support system
Supported Languages:
Go, JavaScript, Java, Python, Objective-C, C++
![Page 12: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/12.jpg)
Let‘s get concrete .. Distributed Tracing
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 12
Use cases for opentracing:
• Tracing Client Calls
• Tracing Server Endpoints
• Tracing a Function
• In-Process Request Context Propagation
• Using Baggage / Distributed Context Propagation
• Logging Events
• …
http://opentracing.io/documentation/pages/instrumentation/common-use-cases.html
Tracer tracer = ...
Span parentSpan = tracer.buildSpan("DoWork") .start();
Span http = tracer.buildSpan("HandleHTTPRequest")
.asChildOf(parentSpan.context())
.withTag("user_agent", req.UserAgent)
.withTag("lucky_number", 42) .start();
Tracer tracer = ...
Span clientSpan = ...
TextMap httpHeadersCarrier = new AnHttpHeaderCarrier(httpRequest);
tracer.inject(span.context(), Format.Builtin.HTTP_HEADERS, httpHeadersCarrier);
Tracer tracer = ...
TextMap httpHeadersCarrier = new AnHttpHeaderCarrier(httpRequest);
SpanContext spanCtx = tracer.extract(Format.Builtin.HTTP_HEADERS, httpHeadersCarrier);
tracer.buildSpan('...').asChildOf(spanCtx).start();
Create
Inject
Extract
OpenTracing API
examples for Java
![Page 13: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/13.jpg)
Let‘s get concrete .. Distributed Tracing
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 13
OpenTracing used in inspectIT
• inspectIT is our open source Application Performance Management solution for monitoring and analyzing
Java(EE) applications. (http://www.inspectit.rocks/)
• inspectIT provides set of remote sensors that enable to trace and correlate calls made between JVM
nodes in your application based on OpenTracing (HTTP and JMS)
• Such traces can provide end-to-end view on the user request execution even if it spans over multiple
JVMs.
DEMO
clients
http
MQ listener
![Page 14: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/14.jpg)
Open Specifications
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 14
Some aspects ...
• Open specifications for metrics
• i.e. Software Metrics Meta-Model™ (SMM™)
• Why not used by commercial APM vendors?
• Open specifications for traces
• i.e. specification blueprint: OPEN.xtrace
• opentracing.io
• Open specifications for monitoring agents and APM solutions
• i.e. inspectIT, Kieker, etc.
Consortiums like SPEC are needed to refine and drive these open specifications and standards!
Analytics
Aggregation
Core/Collection
![Page 15: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/15.jpg)
Let‘s move on!
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 15
• More and more open APM initiatives and tools
• Let‘s combine knowledge and effort to embrace data exchange
• What about an universal specification for an open source Java monitoring agent?
• Maybe one day part of every Java JVM
• Take inspiration from existing and open source tools
• Disrupt the APM vendor market
• Not build the 100th Java agent
• Shift development focus from low-level (agents/data collectors) to high-level (analytics + business monitoring)
• Beneficial for everyone
• Sustainability, independance, transparency, extensibility and speed
• Open solutions
• Commercial vendors
• End user
![Page 16: DISRUPTION OF THE APM MARKET THROUGH OPEN …€¦ · Let‘s get concrete .. OPEN.xtrace 9 10.05.2017 Disruption of the APM market through open standards and open source - our vision](https://reader036.vdocument.in/reader036/viewer/2022070710/5ec3df6493d31c795707cbcb/html5/thumbnails/16.jpg)
THANK YOU
09. MAI 2017
SOFTWARE PERFORMANCE MEETUP
Christian Vögele
Senior Consultant
Application Performance Management
NOVATEC // Consulting GmbH
Landshuter Allee 8-10 // D-80637 München
mobile: +49 171 8193623
fax: +49 711 22040-899
e-mail: [email protected]
web: www.novatec-gmbh.de