the future of apm and why it requires analytics everywhere!

Presenter Name, Title and or DateThe Future of APM…and why it requires analytics everywhere!Patrick Lightbody, VP, Product Management@plightbo ©2008-15 New Relic, Inc. All rights reserved.  

©2008-15 New Relic, Inc. All rights reserved.  

ApplicationPerformanceManagement

©2008-15 New Relic, Inc. All rights reserved.  

Are the servers ok?

©2008-15 New Relic, Inc. All rights reserved.  

Average CPU Utilization

©2008-15 New Relic, Inc. All rights reserved.  

Average CPU Utilization

©2008-15 New Relic, Inc. All rights reserved.  

Database servers

Web servers

Per Server Load Average

©2008-15 New Relic, Inc. All rights reserved.  

East Coast

West Coast

©2008-15 New Relic, Inc. All rights reserved.  

Dimension ExamplesHost ecomm-ws-1.acmecorp.com

Cluster ecomm-web-server, ecomm-db, etc.

Region East Coast, West Coast, etc.

Application Storefront, Inventory Service, etc.

Instance Type Micro, Small, Large, etc.

Availability Zone east-coast-1, west-coast-2, etc.

Operating System Linux vs. Windows (with version)

Purpose Internal vs. External

Attributes

Is the software ok?

©2008-15 New Relic, Inc. All rights reserved.  

Application Response Times (ms)

©2008-15 New Relic, Inc. All rights reserved.  

Transaction Response Times (ms)

©2008-15 New Relic, Inc. All rights reserved.  

Other

Checkout

Login

Add to Cart

Checkout Transaction Breakdown (ms)

©2008-15 New Relic, Inc. All rights reserved.  

REST

Database Cach

eJava

©2008-15 New Relic, Inc. All rights reserved.  

Dimension ExamplesTransaction Name Checkout, Add To Cart, Login, etc.

Total Response Time (total time spent running the transaction)

Tier Response Time 152ms calling a REST service, 593ms calling the DB, etc.

Response Code 404 Not Found, 500 Server Error, etc.

Error Message NullPointerException

Transaction Type Page View, Ajax, Message Queue, etc.

HTTP Verb POST vs. GET

Browser Chrome, Safari, IE, Mobile or Desktop?

More attributes

Are the customers ok?

©2008-15 New Relic, Inc. All rights reserved.  

©2008-15 New Relic, Inc. All rights reserved.   By Andrew Malone (Blank stare Uploaded by Markos90) [CC BY 2.0], via Wikimedia Commonshttps://commons.wikimedia.org/wiki/File%3ABlank_stare.jpg

Customer Breakdown

©2008-15 New Relic, Inc. All rights reserved.  

REST

Database Cach

eJava

Mary buys 2 green widgets

Carlos buys 3 blue

widgets

Anita buys 1 green widget

Acme Corp. buys 10,000 purple

widgets

Customer Breakdown

©2008-15 New Relic, Inc. All rights reserved.  

REST

Database Cach

eJava

Mary buys 2 green widgets

shipping to Boston

Carlos buys 3 blue widgets

shipping to Portland

Anita buys 1 green widget

shipping to London

Acme Corp. buys 10,000 purple

widgets shipping to Dallas

Customer Breakdown

©2008-15 New Relic, Inc. All rights reserved.  

REST

Database Cach

eJava

Mary buys 2 green widgets

shipping to Bostonusing her credit card

Carlos buys 3 blue widgets

shipping to Portlandusing his credit card

Anita buys 1 green widget

shipping to Londonusing a gift card balance

Acme Corp. buys 10,000 purple

widgets shipping to Dallas

using a corporate P.O.

©2008-15 New Relic, Inc. All rights reserved.  

Even more attributes

©2008-15 New Relic, Inc. All rights reserved.  

Dimension ExamplesCustomer ID “Mary” or email address or 823562

Account Name Acme, Globex, Initech, Hooli, etc.

Customer Type Free, Trial, Paid, Premium, etc.

Item Quantity …

Item Details Color, Size, etc.

Cart Value $100 or S, M, L, XL

Shipping Address City, State, Zip Code, Country, etc.

Payment Mechanism Credit card type, gift card, PO, etc.

©2008-15 New Relic, Inc. All rights reserved.  

Virtually every business

is now asoftware business

Managing Software Requires Context

©2008-15 New Relic, Inc. All rights reserved.  

©2008-15 New Relic, Inc. All rights reserved.  

Relational DataApp Server

Customer Web ServerBrowser

23

Public Cloud

NoSQL data store

Private Cloud

API Micro Services

Private Cloud

Application Data

Business DataCustomer Experience

Data

Mobile / Apps / Browser

Managing Software has become a

Big Data Problem

620,448,401,733,239,439,360,000

©2008-15 New Relic, Inc. All rights reserved.  

Managing software requires LOSSLESS data

©2008-15 New Relic, Inc. All rights reserved.  

OR

Managing software requires DIMENSIONAL data

©2008-15 New Relic, Inc. All rights reserved.  

▪App service/transaction, time ▪Error Code▪Instance Type, Server,

Container▪Script, Error, Page, Location,

Server

• App service/transaction, Time

• Geography, Network• Browser, Action, Page,

Session• Device, Crash, Action,

Session

▪Customer ID, New/Existing, Type▪Product ID, Category Type, Content

Type▪Revenue, Cart Size, Traffic, Ad

Impressions

24 dimensions =620 sextillion

different combinations

BusinessSuccess

ApplicationPerformance

CustomerExperience

Confidential ©2008-15 New Relic, Inc. All rights reserved. FOR INTERNAL USE ONLY 27

Managing software requires data at SCALE

Time

Dat

aLossless business outcomes

Lossless customer experienc

e

Lossless performanc

e

Lossy customer

experience

Lossy app performanc

e

Confidential ©2008-15 New Relic, Inc. All rights reserved. FOR INTERNAL USE ONLY 28

Managing software requires ITERATIVE exploration

Dive into

root of cause

Start here➔

Customers report bad experience

Which companies?

Which individuals?

Revenue impact?

How slow?

Bottlenecked at frontend, network, or backend?

Different by geography?

Different by device?Slow frontend assets?

Network bottlenecks?

Which backend app?

Which portion of backend code?

Database bottlenecks?

Backend error?

Dev

Confidential ©2008-15 New Relic, Inc. All rights reserved. FOR INTERNAL USE ONLY 29

Managing software requires COMMON DATA

Eliminate silos for a collaborative, data-driven organization.

OpsProductSoftware

Data

Confidential ©2008-15 New Relic, Inc. All rights reserved. FOR INTERNAL USE ONLY 30

Managing software requires software analytics

How are customers engaging?

Am I open for business

How is my business doing? Lossless data

With many dimensions

Can be iteratively explored

Common across teamsApplicationPerformanc

e

CustomerExperienc

e

BusinessSuccess

Secure, Elastic, Multi-tenant Cloud Solution

©2008-15 New Relic, Inc. All rights reserved.  

Secure, Elastic, Multi-tenant Cloud Solution

Analytics Everywhere!

©2008-15 New Relic, Inc. All rights reserved.  

Announcing… every Pro & Enterprise customer of APM, Browser or Mobile gets Insights with 8 Days of data retention

©2008-15 New Relic, Inc. All rights reserved.  

©2008-15 New Relic, Inc. All rights reserved.  

©2008-15 New Relic, Inc. All rights reserved.  

36

MAKING VIDEO ADS PERSONAL

Tony ChongVP Engineering

37

Who We are?

- Vungle's mission is to ensure no two users have the same video ad experience

- 6 Countries, 160+ Employees- Android and iOS SDK- Node.js backend- 1000+ EC2 instances in AWS- 2.5B requests per day

38

New Relic

- APM for application health- Initially used for 1 QA instance of our Dashboard tool,

moved to Prod fleet in a few weeks- Goto tool for Production issues

- Insights for traffic intelligence- What type of data do we have?- How many types of requests are we serving per day?

per month?- Is there something wrong?

39

APM

- Response Time- Deployments- Error Rate- Throughput- Transactions

40

41

Insights

- Custom Dashboards- NRQL- Different perspectives

42

Insights - Custom Dashboards

43

Insights - Custom Dashboards

44

Insights - NRQL

45

Insights - Different Perspectives

Jason DominaSr. Director, Technical Operations

All content copyright © CafePress. All rights reserved.4747

Who are we?• Created ‘Print On Demand’ in 1999• High volume retail site• Average 600,000 sessions daily• But peak at 4-5 million during the holidays

• If you’re interested/passionate about something, we’ve got a t-shirt, mug, etc… for it. Lot’s of them in fact.

• No, seriously, like millions of them.

All content copyright © CafePress. All rights reserved.4848

CORGI-ZILLA

All content copyright © CafePress. All rights reserved.4949

We care a lot about who’s viewing what

All content copyright © CafePress. All rights reserved.5050

How Fast are we going? Where, Who, How?

All content copyright © CafePress. All rights reserved.5151

Whoa, what happened?

All content copyright © CafePress. All rights reserved.5252

Wait, there’s a difference between Mac and PC users?

©2008-15 New Relic, Inc. All rights reserved.