master’s thesis – final presentation · master’s thesis –final presentation assessing the...

Chair of Software Engineering for Business Information Systems (sebis)

Faculty of Informatics

Technische Universität München

wwwmatthes.in.tum.de

Master’s Thesis – Final Presentation

Assessing the cost and benefit of a microservice landscape

discovery method in the automotive industry

Advisor: Martin Kleehaus, M.Sc.

Student: Nektarios Machner, B.Sc. 04.11.2019

1

Agenda

2

1. Motivation & Problem Description

2. Concept & Foundations

3. Implementation

4. Evaluation

5. Live Demonstration

6. Conclusion

7. Discussion

Agenda

3



3. Implementation

4. Evaluation


6. Conclusion

7. Discussion

4

source:



EAM aims to document and manage the complexity of the business IT landscape in relation to business requirements

Enterprise Architecture Documentation (EAD) challenges:

• time-consuming process

• mostly performed manually

• data incomplete and/or outdated

• lack of clear responsibilities

• IT landscape constantly changing

→ Overcome challenges viaautomated documentation

- majority of organizations have no dedicated process for EA documentation defined

- only 23 participants (18.7%) stated that they have implemented some form of automated EA documentation mechanisms for their EA tool(mostly limited to simple file import mechanisms that are manually triggered)

5

source:

Farwick, M., Hauder, M., Roth, S., Matthes, F., Breu, R.: Enterprise Architecture Documentation: Empirical Analysis of Information Sources for Automation

- In the Hawaii International Conference on System Sciences (HICSS 46), Maui, Hawaii, 2013

Fig.: Usage and relevance as EA information sources (n=123).

7672

6863

59

5154

5855

33 33

39

15

7 712 11

25

0

10

20

30

40

50

60

70

80

NetworkScanners and

Monitors

CMDB PPM ESB ChangeManagement

Tool

LicenseManagement

Tool

Usage of Tool Relevant Info for EA? I don't know

62%

44% 45%47%

27% 27%

32%

20%

55%59%

51%

48%

41%

10%12%

6% 6%9%


6

Which IT artifacts and their communication relationships can be discovered through runtime data?

What are benefits and limitations of this solution?

RQ1

RQ2

1. Motivation & Problem Description – Research Questions

Agenda

7



3. Implementation

4. Evaluation


6. Conclusion

7. Discussion

2. Concept & Foundations – ArchiMate

8

source:

https://www.opengroup.org/togaf

T. O. Group and V. H. Publishing. ArchiMate 3.0.1 Specification. 1st ed. Zaltbommel, Netherlands: Van Haren, 2017. isbn: 978-9-401-80235-2.

Mapping between TOGAF ADM and ArchiMate language

ArchiMate core framework

2. Concept & Foundations – Microlyze

9

source:

Kleehaus, M.; Hauder, M.; Uludag, O.; Corpancho, N.; Matthes, F.: IT Landscape Discovery via Runtime Instrumentation for Automating Enterprise Architecture Model Maintenance,

The Americas Conference on Information Systems (AMCIS), Cancun, Mexiko, 2019.

Project

Management

Version

ControlCMDB WIKI …

Development Environment

Monitoring

Probe

json config

file

<Naming convention>

Enterprise/IT Architects

Federated Information Systems

JSON

schemaValidate JSON schema

against JSON config file

Ap

plic

ati

on

an

d In

fras

tru

ctu

re L

aye

rD

eve

lop

me

nt

Pro

ce

ss

Continuous Integration Pipeline

MICROLYZE

Reference

Repository

IT landscape

Discovery

GraphQL

json config file

Develop-

mentCommit Test Release

Deploy/containerize

transfer

Reference

Frontend

Monitoring

Server

Container

Monitoring

Probe

Micro-

service

Cloud/On-premise Environment

Runtimedata

Monitoring Stream

PO / Developer

maintain

Agenda

10



3. Implementation

4. Evaluation


6. Conclusion

7. Discussion

3. Implementation – Environment

11

source (logos):

www.bmwgroup.com, www.dynatrace.com

Industry partner: BMW

• Department DE-810 (“vehicle data connectivity”)

• 200 BMW employees + external contractors → ~ 500 - 600 employees

• separated into agile teams

• mainly responsible for ConnectedDrive platform

(backend for multiple services in the context of connected cars)

• mostly custom software

• Monitoring Tool in use: Dynatrace AppMon (April 2018)

• Monitoring scope: DE-81, DE-82, partially DE-83

• 100% manual documentation

• no centralized documentation of microservices at all

3. Implementation – Requirements Analysis

12

Rank Artifact Score

1 Data flow and dependencies between applications 18

2 Interfaces / APIs 18

3 Mapping and associations within application layer 18

4 Application Components (logical unit) 17

5 Communication technology (protocols) 17

6 Business Processes 15

7 Mapping and associations within infrastructure layer 15

8 Physical IT resources 14

9 Mapping and associations within business layer 13

10 Use Cases 13

source (logos): www.bmwgroup.com

3. Implementation – Overview

13

source (logos):

www.bmwgroup.com, www.dynatrace.com, www.nodejs.org, www.graphql.org, www.mongodb.com, www.reactjs.org, google.com, developers.google.com/web/tools/puppeteer,

products.office.com/excel, yworks.com/products/yfiles-for-html

Implementation EnvironmentProposed Solution

Backend

Frontend

Monitoring Server

Manual

Documentation Database

Microservice

Monitoring

Agent Discovery Component

Visualizations

3. Implementation – Monitoring Tool (Dynatrace AppMon)

14

- name: String

Application

- name: String

- agentRef: String

- technologyInfo: Object

Process (Agent)

- name: String

- hostGroup: String

- site: String

- os: String

- ipAddress: String

Host

- name: String

- description: String

Agent Group

*

*

1 1..*

1

*runsOn

- targetURL: String

- async: String

PurePath

- source: Agent

- target: Agent

- targetURL: String

Transaction Flow

source

target

1..* *

1..* *

1..*

*

0..1 0..1

source (logos): www.dynatrace.com


15source (logos): www.dynatrace.com

Relevant data structures:

• Transaction flow

• PurePath

Transaction Flow: PurePath:


16source (logos): www.dynatrace.com


17source (logos): www.dynatrace.com, google.com, developers.google.com/web/tools/puppeteer

Limitations & Workarounds:

• Lack of "useful" APIs

• Automation

• Lack of applicable filters

• Naming convention

• Timeframe restrictions

• Completeness of data

• Parameters in requests

• No distinction regarding origin of requests

• Overly strained AppMon instance

login using puppeteer

extract data from cookies

request data (JS fetch API)

process JSON response

3. Implementation – Data Model

18

AppMonApplication

Host

PurePath

Agent Group (Microservice)

Application

Interface

Site

Product

Domain

Client (n/a)

0..1

*

0..1

*

0..1

*

2..*

*

2..*

*

*

*calls

1

*

1..*

*

1..*

1

1..*

*

calls

*

*

subProduct *

1

subDomain *

1

Manual

Documentation


3. Implementation – Data Model

19

Application Collaboration

Node

Application Interaction

Application Component

Business Service

Application Service

Facility

Product

Domain

Device

0..1

*

0..1

*

0..1

*

2..*

*

2..*

*

*

*calls

1

*

1..*

*

1..*

1

1..*

*

calls

*

*

subProduct *

1

subDomain *

1

3. Implementation – Automated Architecture Discovery Algorithm (AADA)

20source (logos): www.bmwgroup.com, dynatrace.com, google.com, developers.google.com/web/tools/puppeteer, products.office.com/excel

Implementation Environment

Monitoring Server

Manual

Documentation

Microservice

Monitoring

Agent

continueAutoArchDiscovery

Entry Point / Controller Data Extraction & Processing

backwards

Discovery

Recursive

Controllers

forwards

Discovery

startAutoArchDiscovery

handle

PurePaths

saveArchitectureToDB

FilteredByApplications

saveBusinessLayer

Information

1 2

34

5

6

Agenda

21



3. Implementation

4. Evaluation


6. Conclusion

7. Discussion

4. Evaluation – Quantitative Analysis

22

Discovery Run:

• Start: September 12th

• Timeframe: 6 hours

• Iterations: 174 (112 backwards, 62 forwards)

i.e. 28 days backwards, 15.5 days forwards

• Duration per iteration: 60 – 120 minutes

Findings:

• High robustness

• No more PurePaths roughly 10 days into the past

• Increasing response times the further back in time

• coverage / accuracy:

221 of 407 Microservices discovered → ~54%

79 of 179 applications “discovered” → ~44%

Discovered Elements:

• Structural:

Application Components: 221

Nodes: 5805

Application Collaborations: 73

Annotations: 14991

• Relationships:

Hierarchy: 12445

Grouping: 2250

Communication: 3031

• Business Layer Elements:

(Sub)Domains: 4

(Sub)Products: 46

Business Services: 79


23

180

185

190

195

200

205

210

215

220

225

Am

ount

Iteration

Discovered Architecture Elements over Time

Application Components


24

0

1000

2000

3000

4000

5000

6000

7000

-112

-109

-106

-103

-100

-97

-94

-91

-88

-85

-82

-79

-76

-73

-70

-67

-64

-61

-58

-55

-52

-49

-46

-43

-40

-37

-34

-31

-28

-25

-22

-19

-16

-13

-10 -7 -4 -1

115

118

121

124

127

130

133

136

139

142

145

148

151

154

157

160

163

166

169

172

Am

ount

Iteration


Nodes


25

0

500

1000

1500

2000

2500

-112

-109

-106

-103

-100

-97

-94

-91

-88

-85

-82

-79

-76

-73

-70

-67

-64

-61

-58

-55

-52

-49

-46

-43

-40

-37

-34

-31

-28

-25

-22

-19

-16

-13

-10 -7 -4 -1

115

118

121

124

127

130

133

136

139

142

145

148

151

154

157

160

163

166

169

172

Am

ount

Iteration


Application Interactions Communication Relationships

4. Evaluation – Qualitative Analysis

26

Question 1:

To what extent do you accept the stated problem description? Do you differ in opinion?

Feedback:

• fully support the problem description

• time and budget restrictions → documentation is not a priority

• data outdated and incomplete indeed true

• lack of clear responsibilities for documentation not true

• uncertainty whether problem is caused by lack of appropriate tools, lack of interest or lack of responsibility

• doubt that manual documentation is actually that time-consuming

(compared to other activities manual documentation does not take that much time relatively seen)


27

Question 2:

How do you rate the approach of extracting architecture information from runtime data in order to assist the IT landscape documentation? What advantages and disadvantages do you see?How do you rate the approach on a scale from 1 (very good) to 5 (very bad)?

Feedback:

• usage of runtime data extremely important

• medium to long-term no other way

• represents the truth

• certain inaccuracy always present

• important but insufficient (lack of explanation)

• understanding of architecture not possible

• connection to source code desirable

• ∅ Grade[n=5] = 1.8


28

Question 3:

How do you rate the approach of maintaining further relationship information within configuration files? What advantages and disadvantages do you see?How do you rate the approach on a scale from 1 (very good) to 5 (very bad)?

Feedback:

• necessary but not revolutionary

• good approach in relation in relation to other approaches

• every bit of contained information needs to add value

(should not contain unnecessary information to minimize maintenance effort)

• JSON preferable to other formats due to validation ability

• handling of unclear or unknown information possibly an issue

• ∅ Grade[n=5] = 1.6


29

Question 4:

How do you rate the approach of ensuring the maintenance of the configuration files through JSON Schema validation?What advantages and disadvantages do you see?How do you rate the approach on a scale from 1 (very good) to 5 (very bad)?

Feedback:

• indispensable

• no alternative

• contingency plan required

• ∅ Grade[n=5] = 1.4


30

Question 5:

To what extent do you perceive the integration of the approach into a CI/CD pipeline as useful?What advantages and disadvantages do you see?How do you rate the approach on a scale from 1 (very good) to 5 (very bad)?

Feedback:

• best approach to force people to do something

• no alternative

• reliability an absolute requirement

• possibility to arouse hatred

• ∅ Grade[n=5] = 1.4


31

Question 6:

How do you rate the cost-benefit ratio of the approach in general?

Feedback:

• estimated costs relatively low

• estimated benefits tremendous

• benefits difficult to quantify

• BMW context:

monitoring tools already in place → no extra cost

pipeline integration perceived as feasible → no big effort

benefits outweigh costs by far (one person estimated value for root-cause-analysis can reach six to seven figures)


32

Visualizations:

Ranking:

1. App. Interaction View(∅ [n=4] = 1.0)

2. Comparison View(∅ [n=5] = 1.8)

3. Communications View (∅ [n=5] = 2.0)

3. App. Landscape View(∅ [n=5] = 2.0)

4. Table View(∅ [n=5] = 2.2)

5. Bus. Landscape View(∅ [n=5] = 2.4)


33

General feedback / Remarks:

Approach:

• manual documentation (in worst case) outdated the moment it is created → automation invaluable

• holistic approach difficult because of existing legacy systems and standard software

• further linking to business layer required

Tool / Visualizations:

• good choice of visualization framework (built-in layouting impressive)

• more export capabilities (especially to MS Office)

• more colors / filters / search capabilities

• support for planned states

4. Evaluation – Before / After (Requirements Analysis)

34

Rank Artifact

1 Data flow and dependencies between applications

2 Interfaces / APIs

3 Mapping and associations within application layer

4 Application Components (logical unit)

5 Communication technology (protocols)

6 Business Processes

7 Mapping and associations within infrastructure layer

8 Physical IT resources

9 Mapping and associations within business layer

10 Use Cases

4. Evaluation – Before / After (Documentation)

35

Type of documentation:red: manualgreen: automatedgrey: not documented

AppMonApplication

Host

PurePath


Application

Interface

Site

Product

Domain

Client (n/a)

0..1

*

0..1

*

0..1

*

2..*

*

*

1

*

1..*

*

1..*

1

1..*

*

calls

subProduct *

1

subDomain *

1

2..*

*

*

AppMonApplication

Host

PurePath


Application

Interface

Site

Product

Domain

Client (n/a)

0..1

*

0..1

*

0..1

*

2..*

*

*

1

*

1..*

*

1..*

1

1..*

*

calls

*subProduct *

1

subDomain *

1

2..*

*

*calls

*

*

calls

*

Agenda

36



3. Implementation

4. Evaluation


6. Conclusion

7. Discussion

Agenda

37



3. Implementation

4. Evaluation


6. Conclusion

7. Discussion

6. Conclusion

38

Benefits:

• Automation

• Depiction of reality

• EA documentation assistance (long-term replacement?)

Limitations:

• Monitoring tool

• Lack of explanation

• Focus on as-is landscape

Outlook:

• Integration of different APM solutions

• Integration into existing landscapes

• Link to business use cases / scenarios

BMW context:

• solution partly implemented in productive environment

• AADA runs in Jenkins pipeline triggered once a day

• creates export of discovered data

• data used to assist documentation

• automated documentation validated

against manual documentation

→ BMW advisors satisfied






Thank you for your attention!

Discussion

Advisor: Martin Kleehaus, M.Sc.

Student: Nektarios Machner, B.Sc. 04.11.2019

39



Chair of Software Engineering for Business

Information Systems

Boltzmannstraße 3

85748 Garching bei München

Tel +49.89.289.

Fax +49.89.289.17136


Nektarios Machner, B.Sc.

[email protected]

40

http://wwwmatthes.in.tum.de/





BACKUP

41

42

+ _id: ObjectId

+ refID: String

+ key: String

+ value: Any

Annotation

+ _id: ObjectId

+ id: String

+ name: String

+ type:

ArchitectureModelType

+ validFrom: Number

+ validTo: Number

+ lastSeen: Number

ArchitectureModel

*

1..*

1

2

+ _id: ObjectId

+ owner: String

+ source: String

+ target: String

+ type:

RelationshipType

+ validFrom: Number

+ validTo: Number

+ lastSeen: Number

Relationship

Domain

Product

Business Service

Device

Application Component

Application Service

Application Collaboration

Application Interaction

Node

Facility

<<enumeration>>

ArchitectureModelType

1

*

Hierarchy

Grouping

Communication

<<enumeration>>

RelationshipType

master’s thesis – final presentation · master’s thesis –final presentation assessing the...

Documents