disruption of the apm market through open …€¦ · let‘s get concrete .. open.xtrace 9...

DISRUPTION OF THE APM MARKET THROUGH

OPEN STANDARDS AND OPEN SOURCE -

OUR VISION OF AN IDEOLOGICAL CHANGE

09. MAI 2017

CHRISTIAN VÖGELE, STEFAN THIEME, ALEXANDER WERT

SOFTWARE PERFORMANCE MEETUP

History of APM Tools: What happened so far?

10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 2

http://4.bp.blogspot.com/-uEdWYuTHUnU/VMSClSFgTcI/AAAAAAAAAPw/uV2YelL9luc/s1600/3-tier.PNG

https://www.manageengine.com/network-monitoring/images/cpu_memory_disk_monitoring.gif http://i176.photobucket.com/albums/w163/GodDreamsOfVegas/nerd-46422.jpg

IT Department

Status quo: Situation has changed

10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 3

Multi-tier architecture Users throughout organization Tool diversification

Data consumers

Development Operations Business

Multi-Vendor, Multi-Tool

10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 4

Development Operations Business

Tools Tools Tools

APM

Data exchange (cross-tool)

Data aggregation

Big data analytics

Business-specific APM

Licence Costs

Vendor Lock-In

Typical Model: x $ per Agent Scaling?

Weakness of commercial APM tools

Microservices Internet of Things

[Abbildung: http://blog.wso2.com] [Abbildung: Christian Hinkelmann, http://nahverkehrhamburg.de]

Flexibility Interoperability Sustainability

Mobile Revolution

[Abbildung: https://uxmag.com]

Adaption to requirements

of company

Error Correction

Data Sources

Analysis Tools

Other APM Tools

Product is not developed

anymore

Strategy Change of APM Vendor

Disruption in the APM tool market

10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 6

Agent - Tool A

Agent - Tool B

Tool A

Tool B

Tool X

Agent - Tool A

Agent - Tool B

Tool A

Tool B

Tool X

* data mapping + conversion

*

*

* * *

Universal Agent

Universal Agent

Tool A

Tool B

Tool X

data mapping

+ conversion

Universal Agent

Universal Agent

Tool A

Tool B

Tool X

Unified specification

What if?

10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 7

Shift focus from collection to usage

Usage

Data Collection

Data Collection

Data Collection

Today

• Commercial tools provide proprietary data

• Data of different tools hard to exchange

• Data of different tools hard to be aggregated

• Commercial tools hardly integrate with free tools

• Company-wide business monitoring not possible

• Current setups support APM maturity only in parts

• Licensing based on low-level (CPU cores, #JVMs)

• …

Future

• Commercial and free tools use open data formats

• Data exchange allows triage big picture analysis

• Data aggregation allows (big data) analytics

• Commercial tools integrate with free tools

• Aggregated data enables business monitoring

• APM maturity process adoption within enterprises

• Licensing based on medium-level KPIs (i.e. usage)

• …

It‘s about Collaboration

10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 8

Performance

Modeling

Web Performance

Analysis

System &

Resources Monitoring Real User Monitoring

JRat

JMemProf

Low-Level

Performance Profiling

Monitoring &

Application Deep Dive

Let‘s get concrete .. OPEN.xtrace

10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 9

OPEN.xtrace format enables

• data interoperability and exchange between APM tools

• SPE researchers to develop their approaches in a tool-agnostic and comparable manner

• Modelling with the information that is available in leading APM tools today

OPEN.xtrace is a community effort as part of the overall goal to increase interoperability of SPE/APM

techniques and tools.

Agent - Tool A

Agent - Tool B

APM Tool

* Trace adapter

*

*

OPEN.xtrace

Let‘s get concrete .. Distributed Tracing

10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 10

A basic distributed trace:

• Distributed tracing has the challenge to monitor transactions propagating across a distributed system

• Distributed tracing is a difficult task as instrumentation must propagate the tracing context both within

and between processes:

• A lot of engineer effort

• Monkey-patched communication packages

• Inconsistencies across platforms

• Not reasonable to use a single tracing vendor

• Traces are truncated in case no mechanism for trace description and propagation is used

http://opentracing.io/documentation/, https://medium.com/opentracing/towards-turnkey-distributed-tracing-5f4297d1736

A trace is a directed

acyclic graph (DAG) of

"spans": named, timed

operations representing

a contiguous segment of

work in that trace.

Let‘s get concrete .. Distributed Tracing

10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 11

OpenTracing is a open distributed tracing standard

• What to standardize? (Encoding formats and/or standard instrumentation APIs)

• OpenTracing allows developers of application code, OSS packages, and OSS services to instrument

their own code without binding to any particular tracing vendor

• OpenTracing provides API standards and allows the concrete implementations to take control over

encoding formats; this in turn gives the application owner the flexibility to switch (or add) tracing vendor.

https://medium.com/opentracing/towards-turnkey-distributed-tracing-5f4297d1736 OSS: Operations support system

Supported Languages:

Go, JavaScript, Java, Python, Objective-C, C++

Let‘s get concrete .. Distributed Tracing

10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 12

Use cases for opentracing:

• Tracing Client Calls

• Tracing Server Endpoints

• Tracing a Function

• In-Process Request Context Propagation

• Using Baggage / Distributed Context Propagation

• Logging Events

• …

http://opentracing.io/documentation/pages/instrumentation/common-use-cases.html

Tracer tracer = ...

Span parentSpan = tracer.buildSpan("DoWork") .start();

Span http = tracer.buildSpan("HandleHTTPRequest")

.asChildOf(parentSpan.context())

.withTag("user_agent", req.UserAgent)

.withTag("lucky_number", 42) .start();

Tracer tracer = ...

Span clientSpan = ...

TextMap httpHeadersCarrier = new AnHttpHeaderCarrier(httpRequest);

tracer.inject(span.context(), Format.Builtin.HTTP_HEADERS, httpHeadersCarrier);

Tracer tracer = ...

TextMap httpHeadersCarrier = new AnHttpHeaderCarrier(httpRequest);

SpanContext spanCtx = tracer.extract(Format.Builtin.HTTP_HEADERS, httpHeadersCarrier);

tracer.buildSpan('...').asChildOf(spanCtx).start();

Create

Inject

Extract

OpenTracing API

examples for Java

Let‘s get concrete .. Distributed Tracing

10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 13

OpenTracing used in inspectIT

• inspectIT is our open source Application Performance Management solution for monitoring and analyzing

Java(EE) applications. (http://www.inspectit.rocks/)

• inspectIT provides set of remote sensors that enable to trace and correlate calls made between JVM

nodes in your application based on OpenTracing (HTTP and JMS)

• Such traces can provide end-to-end view on the user request execution even if it spans over multiple

JVMs.

DEMO

clients

http

MQ listener

Open Specifications

10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 14

Some aspects ...

• Open specifications for metrics

• i.e. Software Metrics Meta-Model™ (SMM™)

• Why not used by commercial APM vendors?

• Open specifications for traces

• i.e. specification blueprint: OPEN.xtrace

• opentracing.io

• Open specifications for monitoring agents and APM solutions

• i.e. inspectIT, Kieker, etc.

Consortiums like SPEC are needed to refine and drive these open specifications and standards!

Analytics

Aggregation

Core/Collection

Let‘s move on!

10.05.2017 Disruption of the APM market through open standards and open source - our vision of an ideological change 15

• More and more open APM initiatives and tools

• Let‘s combine knowledge and effort to embrace data exchange

• What about an universal specification for an open source Java monitoring agent?

• Maybe one day part of every Java JVM

• Take inspiration from existing and open source tools

• Disrupt the APM vendor market

• Not build the 100th Java agent

• Shift development focus from low-level (agents/data collectors) to high-level (analytics + business monitoring)

• Beneficial for everyone

• Sustainability, independance, transparency, extensibility and speed

• Open solutions

• Commercial vendors

• End user

THANK YOU

09. MAI 2017

SOFTWARE PERFORMANCE MEETUP

Christian Vögele

Senior Consultant

Application Performance Management

NOVATEC // Consulting GmbH

Landshuter Allee 8-10 // D-80637 München

mobile: +49 171 8193623

fax: +49 711 22040-899

e-mail: christian.voegele@novatec-gmbh.de

web: www.novatec-gmbh.de