disruption of the apm market through open …€¦ · let‘s get concrete .. open.xtrace 9...
TRANSCRIPT
DISRUPTION OF THE APM MARKET THROUGH
OPEN STANDARDS AND OPEN SOURCE -
OUR VISION OF AN IDEOLOGICAL CHANGE
09. MAI 2017
CHRISTIAN VÖGELE, STEFAN THIEME, ALEXANDER WERT
SOFTWARE PERFORMANCE MEETUP
History of APM Tools: What happened so far?
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 2
http://4.bp.blogspot.com/-uEdWYuTHUnU/VMSClSFgTcI/AAAAAAAAAPw/uV2YelL9luc/s1600/3-tier.PNG
https://www.manageengine.com/network-monitoring/images/cpu_memory_disk_monitoring.gif http://i176.photobucket.com/albums/w163/GodDreamsOfVegas/nerd-46422.jpg
IT Department
Status quo: Situation has changed
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 3
Multi-tier architecture Users throughout organization Tool diversification
Data consumers
Development Operations Business
Multi-Vendor, Multi-Tool
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 4
Development Operations Business
Tools Tools Tools
APM
Data exchange (cross-tool)
Data aggregation
Big data analytics
Business-specific APM
Licence Costs
Vendor Lock-In
Typical Model: x $ per Agent Scaling?
Weakness of commercial APM tools
Microservices Internet of Things
[Abbildung: http://blog.wso2.com] [Abbildung: Christian Hinkelmann, http://nahverkehrhamburg.de]
Flexibility Interoperability Sustainability
Mobile Revolution
[Abbildung: https://uxmag.com]
Adaption to requirements
of company
Error Correction
Data Sources
Analysis Tools
Other APM Tools
Product is not developed
anymore
Strategy Change of APM Vendor
Disruption in the APM tool market
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 6
Agent - Tool A
Agent - Tool B
Tool A
Tool B
Tool X
Agent - Tool A
Agent - Tool B
Tool A
Tool B
Tool X
* data mapping + conversion
*
*
* * *
Universal Agent
Universal Agent
Tool A
Tool B
Tool X
data mapping
+ conversion
Universal Agent
Universal Agent
Tool A
Tool B
Tool X
Unified specification
What if?
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 7
Shift focus from collection to usage
Usage
Data Collection
Data Collection
Data Collection
Today
• Commercial tools provide proprietary data
• Data of different tools hard to exchange
• Data of different tools hard to be aggregated
• Commercial tools hardly integrate with free tools
• Company-wide business monitoring not possible
• Current setups support APM maturity only in parts
• Licensing based on low-level (CPU cores, #JVMs)
• …
Future
• Commercial and free tools use open data formats
• Data exchange allows triage big picture analysis
• Data aggregation allows (big data) analytics
• Commercial tools integrate with free tools
• Aggregated data enables business monitoring
• APM maturity process adoption within enterprises
• Licensing based on medium-level KPIs (i.e. usage)
• …
It‘s about Collaboration
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 8
Performance
Modeling
Web Performance
Analysis
System &
Resources Monitoring Real User Monitoring
JRat
JMemProf
Low-Level
Performance Profiling
Monitoring &
Application Deep Dive
Let‘s get concrete .. OPEN.xtrace
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 9
OPEN.xtrace format enables
• data interoperability and exchange between APM tools
• SPE researchers to develop their approaches in a tool-agnostic and comparable manner
• Modelling with the information that is available in leading APM tools today
OPEN.xtrace is a community effort as part of the overall goal to increase interoperability of SPE/APM
techniques and tools.
Agent - Tool A
Agent - Tool B
APM Tool
* Trace adapter
*
*
OPEN.xtrace
Let‘s get concrete .. Distributed Tracing
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 10
A basic distributed trace:
• Distributed tracing has the challenge to monitor transactions propagating across a distributed system
• Distributed tracing is a difficult task as instrumentation must propagate the tracing context both within
and between processes:
• A lot of engineer effort
• Monkey-patched communication packages
• Inconsistencies across platforms
• Not reasonable to use a single tracing vendor
• Traces are truncated in case no mechanism for trace description and propagation is used
http://opentracing.io/documentation/, https://medium.com/opentracing/towards-turnkey-distributed-tracing-5f4297d1736
A trace is a directed
acyclic graph (DAG) of
"spans": named, timed
operations representing
a contiguous segment of
work in that trace.
Let‘s get concrete .. Distributed Tracing
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 11
OpenTracing is a open distributed tracing standard
• What to standardize? (Encoding formats and/or standard instrumentation APIs)
• OpenTracing allows developers of application code, OSS packages, and OSS services to instrument
their own code without binding to any particular tracing vendor
• OpenTracing provides API standards and allows the concrete implementations to take control over
encoding formats; this in turn gives the application owner the flexibility to switch (or add) tracing vendor.
https://medium.com/opentracing/towards-turnkey-distributed-tracing-5f4297d1736 OSS: Operations support system
Supported Languages:
Go, JavaScript, Java, Python, Objective-C, C++
Let‘s get concrete .. Distributed Tracing
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 12
Use cases for opentracing:
• Tracing Client Calls
• Tracing Server Endpoints
• Tracing a Function
• In-Process Request Context Propagation
• Using Baggage / Distributed Context Propagation
• Logging Events
• …
http://opentracing.io/documentation/pages/instrumentation/common-use-cases.html
Tracer tracer = ...
Span parentSpan = tracer.buildSpan("DoWork") .start();
Span http = tracer.buildSpan("HandleHTTPRequest")
.asChildOf(parentSpan.context())
.withTag("user_agent", req.UserAgent)
.withTag("lucky_number", 42) .start();
Tracer tracer = ...
Span clientSpan = ...
TextMap httpHeadersCarrier = new AnHttpHeaderCarrier(httpRequest);
tracer.inject(span.context(), Format.Builtin.HTTP_HEADERS, httpHeadersCarrier);
Tracer tracer = ...
TextMap httpHeadersCarrier = new AnHttpHeaderCarrier(httpRequest);
SpanContext spanCtx = tracer.extract(Format.Builtin.HTTP_HEADERS, httpHeadersCarrier);
tracer.buildSpan('...').asChildOf(spanCtx).start();
Create
Inject
Extract
OpenTracing API
examples for Java
Let‘s get concrete .. Distributed Tracing
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 13
OpenTracing used in inspectIT
• inspectIT is our open source Application Performance Management solution for monitoring and analyzing
Java(EE) applications. (http://www.inspectit.rocks/)
• inspectIT provides set of remote sensors that enable to trace and correlate calls made between JVM
nodes in your application based on OpenTracing (HTTP and JMS)
• Such traces can provide end-to-end view on the user request execution even if it spans over multiple
JVMs.
DEMO
clients
http
MQ listener
Open Specifications
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 14
Some aspects ...
• Open specifications for metrics
• i.e. Software Metrics Meta-Model™ (SMM™)
• Why not used by commercial APM vendors?
• Open specifications for traces
• i.e. specification blueprint: OPEN.xtrace
• opentracing.io
• Open specifications for monitoring agents and APM solutions
• i.e. inspectIT, Kieker, etc.
Consortiums like SPEC are needed to refine and drive these open specifications and standards!
Analytics
Aggregation
Core/Collection
Let‘s move on!
10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 15
• More and more open APM initiatives and tools
• Let‘s combine knowledge and effort to embrace data exchange
• What about an universal specification for an open source Java monitoring agent?
• Maybe one day part of every Java JVM
• Take inspiration from existing and open source tools
• Disrupt the APM vendor market
• Not build the 100th Java agent
• Shift development focus from low-level (agents/data collectors) to high-level (analytics + business monitoring)
• Beneficial for everyone
• Sustainability, independance, transparency, extensibility and speed
• Open solutions
• Commercial vendors
• End user
THANK YOU
09. MAI 2017
SOFTWARE PERFORMANCE MEETUP
Christian Vögele
Senior Consultant
Application Performance Management
NOVATEC // Consulting GmbH
Landshuter Allee 8-10 // D-80637 München
mobile: +49 171 8193623
fax: +49 711 22040-899
e-mail: [email protected]
web: www.novatec-gmbh.de