the future of apm and why it requires analytics everywhere!
TRANSCRIPT
Presenter Name, Title and or DateThe Future of APM…and why it requires analytics everywhere!Patrick Lightbody, VP, Product Management@plightbo ©2008-15 New Relic, Inc. All rights reserved.
©2008-15 New Relic, Inc. All rights reserved.
ApplicationPerformanceManagement
©2008-15 New Relic, Inc. All rights reserved.
Are the servers ok?
©2008-15 New Relic, Inc. All rights reserved.
Average CPU Utilization
©2008-15 New Relic, Inc. All rights reserved.
Average CPU Utilization
©2008-15 New Relic, Inc. All rights reserved.
Database servers
Web servers
Per Server Load Average
©2008-15 New Relic, Inc. All rights reserved.
East Coast
West Coast
©2008-15 New Relic, Inc. All rights reserved.
Dimension ExamplesHost ecomm-ws-1.acmecorp.com
Cluster ecomm-web-server, ecomm-db, etc.
Region East Coast, West Coast, etc.
Application Storefront, Inventory Service, etc.
Instance Type Micro, Small, Large, etc.
Availability Zone east-coast-1, west-coast-2, etc.
Operating System Linux vs. Windows (with version)
Purpose Internal vs. External
Attributes
Is the software ok?
©2008-15 New Relic, Inc. All rights reserved.
Application Response Times (ms)
©2008-15 New Relic, Inc. All rights reserved.
Transaction Response Times (ms)
©2008-15 New Relic, Inc. All rights reserved.
Other
Checkout
Login
Add to Cart
Checkout Transaction Breakdown (ms)
©2008-15 New Relic, Inc. All rights reserved.
REST
Database Cach
eJava
©2008-15 New Relic, Inc. All rights reserved.
Dimension ExamplesTransaction Name Checkout, Add To Cart, Login, etc.
Total Response Time (total time spent running the transaction)
Tier Response Time 152ms calling a REST service, 593ms calling the DB, etc.
Response Code 404 Not Found, 500 Server Error, etc.
Error Message NullPointerException
Transaction Type Page View, Ajax, Message Queue, etc.
HTTP Verb POST vs. GET
Browser Chrome, Safari, IE, Mobile or Desktop?
More attributes
Are the customers ok?
©2008-15 New Relic, Inc. All rights reserved.
©2008-15 New Relic, Inc. All rights reserved. By Andrew Malone (Blank stare Uploaded by Markos90) [CC BY 2.0], via Wikimedia Commonshttps://commons.wikimedia.org/wiki/File%3ABlank_stare.jpg
Customer Breakdown
©2008-15 New Relic, Inc. All rights reserved.
REST
Database Cach
eJava
Mary buys 2 green widgets
Carlos buys 3 blue
widgets
Anita buys 1 green widget
Acme Corp. buys 10,000 purple
widgets
Customer Breakdown
©2008-15 New Relic, Inc. All rights reserved.
REST
Database Cach
eJava
Mary buys 2 green widgets
shipping to Boston
Carlos buys 3 blue widgets
shipping to Portland
Anita buys 1 green widget
shipping to London
Acme Corp. buys 10,000 purple
widgets shipping to Dallas
Customer Breakdown
©2008-15 New Relic, Inc. All rights reserved.
REST
Database Cach
eJava
Mary buys 2 green widgets
shipping to Bostonusing her credit card
Carlos buys 3 blue widgets
shipping to Portlandusing his credit card
Anita buys 1 green widget
shipping to Londonusing a gift card balance
Acme Corp. buys 10,000 purple
widgets shipping to Dallas
using a corporate P.O.
©2008-15 New Relic, Inc. All rights reserved.
Even more attributes
©2008-15 New Relic, Inc. All rights reserved.
Dimension ExamplesCustomer ID “Mary” or email address or 823562
Account Name Acme, Globex, Initech, Hooli, etc.
Customer Type Free, Trial, Paid, Premium, etc.
Item Quantity …
Item Details Color, Size, etc.
Cart Value $100 or S, M, L, XL
Shipping Address City, State, Zip Code, Country, etc.
Payment Mechanism Credit card type, gift card, PO, etc.
©2008-15 New Relic, Inc. All rights reserved.
Virtually every business
is now asoftware business
Managing Software Requires Context
©2008-15 New Relic, Inc. All rights reserved.
©2008-15 New Relic, Inc. All rights reserved.
Relational DataApp Server
Customer Web ServerBrowser
23
Public Cloud
NoSQL data store
Private Cloud
API Micro Services
Private Cloud
Application Data
Business DataCustomer Experience
Data
Mobile / Apps / Browser
Managing Software has become a
Big Data Problem
620,448,401,733,239,439,360,000
©2008-15 New Relic, Inc. All rights reserved.
Managing software requires LOSSLESS data
©2008-15 New Relic, Inc. All rights reserved.
OR
Managing software requires DIMENSIONAL data
©2008-15 New Relic, Inc. All rights reserved.
▪App service/transaction, time ▪Error Code▪Instance Type, Server,
Container▪Script, Error, Page, Location,
Server
• App service/transaction, Time
• Geography, Network• Browser, Action, Page,
Session• Device, Crash, Action,
Session
▪Customer ID, New/Existing, Type▪Product ID, Category Type, Content
Type▪Revenue, Cart Size, Traffic, Ad
Impressions
24 dimensions =620 sextillion
different combinations
BusinessSuccess
ApplicationPerformance
CustomerExperience
Confidential ©2008-15 New Relic, Inc. All rights reserved. FOR INTERNAL USE ONLY 27
Managing software requires data at SCALE
Time
Dat
aLossless business outcomes
Lossless customer experienc
e
Lossless performanc
e
Lossy customer
experience
Lossy app performanc
e
Confidential ©2008-15 New Relic, Inc. All rights reserved. FOR INTERNAL USE ONLY 28
Managing software requires ITERATIVE exploration
Dive into
root of cause
Start here➔
Customers report bad experience
Which companies?
Which individuals?
Revenue impact?
How slow?
Bottlenecked at frontend, network, or backend?
Different by geography?
Different by device?Slow frontend assets?
Network bottlenecks?
Which backend app?
Which portion of backend code?
Database bottlenecks?
Backend error?
Dev
Confidential ©2008-15 New Relic, Inc. All rights reserved. FOR INTERNAL USE ONLY 29
Managing software requires COMMON DATA
Eliminate silos for a collaborative, data-driven organization.
OpsProductSoftware
Data
Confidential ©2008-15 New Relic, Inc. All rights reserved. FOR INTERNAL USE ONLY 30
Managing software requires software analytics
How are customers engaging?
Am I open for business
How is my business doing? Lossless data
With many dimensions
Can be iteratively explored
Common across teamsApplicationPerformanc
e
CustomerExperienc
e
BusinessSuccess
Secure, Elastic, Multi-tenant Cloud Solution
©2008-15 New Relic, Inc. All rights reserved.
Secure, Elastic, Multi-tenant Cloud Solution
Analytics Everywhere!
©2008-15 New Relic, Inc. All rights reserved.
Announcing… every Pro & Enterprise customer of APM, Browser or Mobile gets Insights with 8 Days of data retention
©2008-15 New Relic, Inc. All rights reserved.
©2008-15 New Relic, Inc. All rights reserved.
©2008-15 New Relic, Inc. All rights reserved.
36
MAKING VIDEO ADS PERSONAL
Tony ChongVP Engineering
37
Who We are?
- Vungle's mission is to ensure no two users have the same video ad experience
- 6 Countries, 160+ Employees- Android and iOS SDK- Node.js backend- 1000+ EC2 instances in AWS- 2.5B requests per day
38
New Relic
- APM for application health- Initially used for 1 QA instance of our Dashboard tool,
moved to Prod fleet in a few weeks- Goto tool for Production issues
- Insights for traffic intelligence- What type of data do we have?- How many types of requests are we serving per day?
per month?- Is there something wrong?
39
APM
- Response Time- Deployments- Error Rate- Throughput- Transactions
40
41
Insights
- Custom Dashboards- NRQL- Different perspectives
42
Insights - Custom Dashboards
43
Insights - Custom Dashboards
44
Insights - NRQL
45
Insights - Different Perspectives
Jason DominaSr. Director, Technical Operations
All content copyright © CafePress. All rights reserved.4747
Who are we?• Created ‘Print On Demand’ in 1999• High volume retail site• Average 600,000 sessions daily• But peak at 4-5 million during the holidays
• If you’re interested/passionate about something, we’ve got a t-shirt, mug, etc… for it. Lot’s of them in fact.
• No, seriously, like millions of them.
All content copyright © CafePress. All rights reserved.4848
CORGI-ZILLA
All content copyright © CafePress. All rights reserved.4949
We care a lot about who’s viewing what
All content copyright © CafePress. All rights reserved.5050
How Fast are we going? Where, Who, How?
All content copyright © CafePress. All rights reserved.5151
Whoa, what happened?
All content copyright © CafePress. All rights reserved.5252
Wait, there’s a difference between Mac and PC users?
©2008-15 New Relic, Inc. All rights reserved.