end user experience at its best - servicetrace gmbh€¦ · end user experience at its best real...

16
A STUDY WITH CONTRIBUTIONS BY STEFAN DEML, AMASOL AG THOMAS MENDEL, RESEARCH IN ACTION GMBH CHRISTIAN STEINMETZ, SERVICETRACE GMBH White paper End user experience at its best Combining real user and synthec monitoring User experience is becoming increasingly important in a digital society. It can be a critical success factor for businesses.

Upload: others

Post on 25-Jun-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

A STUDY WITH CONTRIBUTIONS BY

STEFAN DEML, AMASOL AG

THOMAS MENDEL, RESEARCH IN ACTION GMBH

CHRISTIAN STEINMETZ, SERVICETRACE GMBH

White paper

End user experience at its bestCombining real user and

synthetic monitoring

User experience is becoming increasingly important in a digital society. It can be a critical success factor for businesses.

Page 2: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

Vendor Selection Matrix Disclaimer: Research in Action GmbH does not endorse any

vendor, product or service depicted in our research publications, and does not advise

technology users to select only those vendors with the highest ratings. The information

contained in this research has been obtained from both enterprise as well as vendor

sources believed to be reliable. Research in Action GmbH’s research publications consist

of the analysts’ opinions and should not be considered as statements of fact. The opin-

ions expressed are subject to change without further notice. Research in Action GmbH

disclaims all warranties, expressed or implied, with respect to this research, including any

warranties of merchantability or fitness for a particular purpose.

© 2018 – Copyright note: All contents of this whitepaper, in particular texts, photo-

graphs and graphics, are protected by copyright. Unless expressly indicated otherwise,

the copyright is held by the Servicetrace GmbH.

Page 3: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

END USER EXPERIENCE AT ITS BEST  – REAL USER + SYNTHETIC MONITORING

3

CONTENTS

� EXECUTIVE SUMMARY ............................................................... 04

� INTRODUCTION .............................................................................. 05

� 2018 APM MARKET OVERVIEW ............................................ 07

Definition, market, requirements ...................................................... 07

Trends .............................................................................................. 08

� USER EXPERIENCE AS A SUCCESS FACTOR ...................... 09

� END USER EXPERIENCE MONITORING: METHODS AND TECHNOLOGIES ......................................... 10

Bottom up vs. top down .................................................................. 10

Bottom up ............................................................................... 10

Top down ................................................................................ 10

Real user vs. synthetic monitoring ................................................... 11

Real user monitoring .............................................................. 11

Synthetic monitoring .............................................................. 11

� END USER EXPERIENCE MONITORING: BEST PRACTICE ............................................................................ 12

Combined: real user and synthetic monitoring ................................. 12

Real user and synthetic monitoring from two leading providers ...... 13

Dynatrace SaaS / Managed .................................................... 13

Servicetrace Robotic Solutions ............................................... 14

Best of breed: the Dynatrace / Servicetrace interface ................... 15

kubus IT case study .......................................................................... 15

Page 4: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

END USER EXPERIENCE AT ITS BEST  – REAL USER + SYNTHETIC MONITORING

4

EXECUTIVE SUMMARY

Our digital society means that user experience is becoming increasingly important to application users and customers. It can be a critical success factor for a business and should be a key priority in IT management.

Two methods have become established for ascertaining the performance-related parameters of the user experience. These are i) real user monitoring – the passive monitoring of real user transactions – and ii) synthetic monitoring – the active monitoring of applica-tions by simulated (synthetic) users.

Combining these methods brings together the advantages of both technologies and makes it possible to obtain a complete picture of the availability and performance of ap-plications. This enables service quality, and therefore user experience, to be measured objectively and improved in a systematic way with the goal of continuous service opti-mization for users, customers, and IT service management alike.

Consultancy firm Research in Action has assessed an exist-ing integrated solution provided by two leading vendors in the fields of real user monitoring and synthetic monitoring. They found it to be a technologically mature solution that is also easy to implement.

Page 5: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

END USER EXPERIENCE AT ITS BEST  – REAL USER + SYNTHETIC MONITORING

5

INTRODUCTION

Monitoring complex applications is becoming increasingly difficult. Years ago, applications were run centrally on mainframe computers. It was easy to moni-tor this manageable one-tier architecture and the causes of problems could be quickly identified and resolved.

Following on from that, client-server applications were introduced with com-ponents distributed over two to three tiers. Typically this kind of application infrastructure consisted of a web server, an application server, and a database server.

Error analysis started to require more effort because the various components were usually looked after by different departments. This made effective com-munication during the analysis more difficult.

The term ‘silo thinking’ emerged to describe a lack of teamwork and the inad-equate transparency between different parts of the business and the problems this caused.

Application monitoring becomes more demanding as the complexity of IT environ-ments increases

Page 6: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

END USER EXPERIENCE AT ITS BEST  – REAL USER + SYNTHETIC MONITORING

6

These days, applications extend across multiple tiers and possibly multiple loca-tions across the globe. Business applications depend on other IT services such as DNS, LDAP, network infrastructure, third-party services, etc. Applications are based on different technologies and are run on virtual servers offering high availability features. This makes it possible to shift an application from one data center to another on the fly. IT services may be operated entirely in the cloud or run as hybrid cloud solutions. In short, modern IT environments are complex and highly dynamic operation is standard.

Even extensive monitoring solutions covering individual components at all levels frequently prove inadequate due to the heterogeneous architecture and dynamic nature of IT environments. Despite the separate monitoring tools all reporting a ‘green’ status, the end user nevertheless faces application downtime or perform-ance problems.

Monitoring individual com-ponents fails to reflect the IT service qual-ity experienced by users

Bottom-up approaches that provide monitoring at business service level, e.g. where services are modeled based on the individual components, often cannot keep pace with the dynamic nature of present IT infrastructures. It is therefore becoming more and more important to complement infrastructure monitoring with effective end user monitoring in order to examine IT services from the user’s perspective.

Page 7: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

END USER EXPERIENCE AT ITS BEST  – REAL USER + SYNTHETIC MONITORING

7

Research in Action identifies end user moni-toring as one of most important requirements in the APM market

SourceAPM Vendor Selection Matrix 2018 >> bit.ly/2LmGFEQ

2018 APM MARKET OVERVIEW

Definition, market, requirements

Application Performance Management (APM) is defined as the monitoring, control and management of the performance and availability of software appli-cations. The aim of APM is to detect complex performance issues experienced by applications, perform diagnostics, and resolve any issues in order to deliver a defined level of service quality.

Application Performance Management represents a highly dynamic market in which technology is continually evolving and that drives innovation. The 2018 APM Vendor Selection Matrix compiled by consultancy firm Research in Action identifies the top 20 next-generation Application Performance Manage-ment solutions based on the results of a survey of 1,500 IT decision-makers in medium-sized and large enterprises, worldwide.

The requirements and expectations that customers have of APM solutions have changed significantly over the last ten years. That is why, according to the latest survey, the requirement for end-to-end monitoring is now of consider-able importance.

Page 8: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

END USER EXPERIENCE AT ITS BEST  – REAL USER + SYNTHETIC MONITORING

8

The APM Ven-dor Selection Matrix pos- itions Dyna-trace and Ser-vicetrace in the top 5 of next generation APM providers

SourceAPM Vendor Selection Matrix 2018 >> bit.ly/2LmGFEQ

Trends

Because IT applications are becoming increasingly complex, it is getting more and more difficult to measure end user response times. At the same time, IT service quality is evolving into a central, business-critical factor, which increases the need for reliable, precise monitoring of user experience and end-to-end transactions.

Research in Action examined best practice for comprehensively and seam-lessly quantifying the end user experience. They found methodically combining synthetic monitoring and real user monitoring to be an important market trend. Integrating both approaches gives a complete and detailed picture of applica-tion performance from the user perspective and provides the basis for continu-ous service optimization.

As part of its exploration of trends in the field of APM, the consultancy firm took a closer look at an innovative solution: Dynatrace real user monitoring and synthetic monitoring from Servicetrace. The combined end user monitor-ing solution stands out due to its ease of implementation and the high tech-nological maturity delivered by both providers. Research in Action lists both Dynatrace and Servicetrace in the top five “next-generation Application Per-formance Management solution providers”.

Page 9: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

END USER EXPERIENCE AT ITS BEST  – REAL USER + SYNTHETIC MONITORING

9

USER EXPERIENCE AS A SUCCESS FACTOR

In an increasingly digital society, user experience has emerged as the number one business-critical factor. From web stores to CRM and ERP, a company’s success depends significantly on the user experience of customers, both in-ternal and external. This is because a positive user experience correlates with high customer satisfaction and increased productivity.

The user experience (UX) encompasses all aspects of a user’s interaction with a service or product, for example digital applications.

In a digital culture, the user experience is a fundamental success factor for any business

The key criteria for evaluating the user experience are:

� Usability How user-friendly applications are

� Availability / performance of applications

This study focuses on the second criterion, particularly the use of perform-ance-related parameters for determining the user experience.

Page 10: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

END USER EXPERIENCE AT ITS BEST  – REAL USER + SYNTHETIC MONITORING

10

USER EXPERIENCE MONITORING: METHODS AND TECHNOLOGIES

Bottom up vs. top down

Bottom up – Companies use a bewildering array of monitoring solutions for analyzing and optimizing the health status of business-critical applications. Many conventional solutions are based on monitoring all the individual com-ponents in an IT stack, ranging from server infrastructure, through application tiers (e.g. databases, middleware, and web servers), to network components. A bottom-up approach attempts to draw conclusions about the status of the business service available to the end user or customer by combining all this separately collected measurement data. The process is very laborious and most problems with applications are resolved reactively as a result. With IT becom-ing increasingly complex and dynamic, it is ever more difficult to assign specific monitoring information to a business service. The torrent of alerts generated at individual component level is scarcely manageable, leading to situations where alerts are usually ignored. This data is, however, very pertinent and helpful when it comes to troubleshooting and quickly performing root cause analysis when problems arise.

Top down – Effective top-down, end-to-end monitoring is required to meet the demands of proactive monitoring. This is achieved by accurately and ob-jectively measuring application availability and performance at business service level. Services provided to the user are measured and tested from the user perspective. Two practical methods have emerged in this area: real user monitoring and synthetic monitoring.

Page 11: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

END USER EXPERIENCE AT ITS BEST  – REAL USER + SYNTHETIC MONITORING

11

Real user vs. synthetic monitoring

Real user monitoring (RUM) records all the interactions that genuine users have with web-, server-, and cloud-based applications. Analysis of the real user data collected shows the application service quality that users are currently experiencing, highlighting when there is downtime or performance issues.

Real user data provides valuable information for troubleshooting and/or root cause analysis when there are problems with the application(s). As a passive monitoring technology, real user monitoring only collects data when users are active in the system and carrying out transactions.

Synthetic monitoring deploys simulated users in the form of software robots. They perform predefined, recurring user transactions within an application 24/7, measuring availability and performance at the same time. Constant operation of these ‘digital users’ enables proactive monitoring by continuously monitoring service availability. It also allows compliance with service level agreements to be demonstrated even when there is no real user activity.

Unlike human users, who interact with applications selectively and inconsist-ently, synthetic users repeatedly follow standardized workflows with pre-defined reference measurement points. This means that all application trans-actions that are prone to malfunction can be tested at regular intervals. Being able to standardize a sequence of actions also makes it possible to perform objective comparisons of application performance, for example at different locations.

Page 12: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

END USER EXPERIENCE AT ITS BEST  – REAL USER + SYNTHETIC MONITORING

12

USER EXPERIENCE MONITORING: BEST PRACTICE

The combination of real user analytics and synthetic monitoring

Using real user or synthetic monitoring in isolation delivers information that is not sufficiently complete to allow service quality to be optimized to meet the demands of today’s customers and users.

Only by combining the two technologies can a broad enough range of data be generated for a truly comprehensive analysis of the IT service quality experi-enced by users.

If information about the end user experience is then correlated with data derived from classic application and infrastructure monitoring it is possible to obtain a complete picture of the application landscape.

This in turn enables detailed error analysis and fault resolution to be targeted with pinpoint precision if availability or performance issues arise.

Best practice for comprehen-sive analysis of the end user experience: a twin-track ap-proach combin-ing real user and synthetic monitoring

Page 13: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

END USER EXPERIENCE AT ITS BEST  – REAL USER + SYNTHETIC MONITORING

13

Real user and synthetic monitoring from two leading providers

Dynatrace SaaS / Managed – The Dynatrace platform automatically detects and models links between IT components in dynamic and complex environ-ments. This allows root cause analysis to be conducted quickly in the event of a fault. It provides comprehensive visualization across the whole IT stack associated with an application, from the end user’s perspective, through appli-cation services, to the underlying IT infrastructure at the data center or cloud provider.

Dynatrace Real User Monitoring collects application metrics from user sessions and correlates this data with information about the underlying infrastructure and applications. Artificial intelligence components help resolve problems in dynamic and complex environments.

Dynatrace can be fully automated using an open API so that no intervention is necessary, be it during deployment, configuration, analysis or fault resolution.

The Dynatrace SaaS / Man-aged platform offers compre-hensive applica-tion perform-ance analysis across all tiers

Page 14: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

END USER EXPERIENCE AT ITS BEST  – REAL USER + SYNTHETIC MONITORING

14

Servicetrace Robotic Solutions – As ‘synthetic users’, Servicetrace software robots operate each user interface in exactly the same way as a human user using the same means, e.g. image recognition, mouse clicks, and keyboard inputs.

This universal procedure can be used to automate absolutely any Windows-based application and to monitor availability and performance at transaction level 24/7.

Although the sequences of transactions (Workflows) performed by the robot run on the user interface (GUI), they are performed in the background as a hid-den Windows session and are therefore protected against unauthorized access by third parties.

Workflows for the robots are created using Workflow Studio, an intuitive tool that is quick to learn. No programming knowledge is required.

Design Workflows graphically and simulate any user transaction using Service-trace synthetic monitoring

Logon to SAP Hybrid CloudWorkflow Design >> bit.ly/2NhEELU

Robot Run >> bit.ly/2LoGHvY

Page 15: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

END USER EXPERIENCE AT ITS BEST  – REAL USER + SYNTHETIC MONITORING

15

Best of breed end user monitoring: the Dynatrace / Servicetrace interface

Servicetrace robots complement the Dynatrace SaaS / Managed platform by providing synthetic monitoring data via an open API. Synthetic data is seam-lessly integrated into the Dynatrace platform by aligning it with the Dynatrace data model. This enables information from real users and synthetic monitoring to be analyzed together.

The combined solution from these two leading vendors in the fields of real user and synthetic monitoring thus provides a complete, detailed representa-tion of the end user experience. Due to the technical excellence of the two providers, Research in Action judges this integrated approach to be the best solution currently available for delivering service quality in complex and dy-namic IT environments in line with existing customer requirements and service level agreements.

Seamless inte-gration: syn-thetic monitor-ing data from Servicetrace on the Dynatrace dashboard

kubus IT case study – As an IT service provider to health insurers AOK Bayern and AOK Plus, kubus IT looks after 17,000 users in the German states of Bavaria, Thuringia, and Saxony. Their central business pro-cesses run on oscare®, an SAP-based solution for the insurance indus-try. The solution incorporates over 20 system components accessed via Citrix. In order to monitor these highly business-critical, complex ap-plications, kubus IT opted for the twin-track approach described above. >> bit.ly/2C5taKB�

Page 16: End user experience at its best - Servicetrace GmbH€¦ · END USER EXPERIENCE AT ITS BEST REAL USER + SYNTHETIC MONITORING 6 These days, applications extend across multiple tiers

AUTHORS

Dr. Thomas Mendel founded Research in Action GmbH in 2011 after nine years at For-

rester Research. The company is a leading independent research and consultancy firm

in the field of information and communications technology. A renowned IT industry

expert, Mendel has been advising IT vendors and end user clients for over 25 years

and heads up international market research projects. His work combines high-quality

surveys with practical, actionable recommendations.

[email protected] +49 160 99492223 www.researchinaction.de

Stefan Deml, co-founder and board member of system integrator and EMEA Dyna-

trace partner amasol AG, has been supporting customers for around 20 years. He

guides clients from requirements definition, through solution selection, design, imple-

mentation, and commissioning, to continuous improvement of implemented systems.

His sharp instinct for what is feasible helps him design pragmatic IT solutions suitable

for any business.

[email protected] +49 171 7742969 www.amasol.de

Christian Steinmetz has 14 years of experience in IT consultancy and implementation

of IT solutions for medium-sized and large enterprises. Software robots, service moni-

toring, and data analytics have been important elements of his work for many years.

As the business consultancy lead at Servicetrace GmbH he advises customers across

a wide range of sectors on robotic process automation and application performance

monitoring. Servicetrace specializes in software robotic solutions.

[email protected] +49 152 019 097 05 www.servicetrace.de