4finance microservice pilot in poland

Microservices

PILOT

Agendaproblems being solvedhow to use it

Problems of a microservice

Problems of a microservicewho can I talk towhere are my collaboratorshow can I contact othershow can others contact mewhat’s my APIhow to be part of a bigger system: logshow do I expose my metricshow do I view my metrics

Who can I talk to?solved by:

4finance micro-infra-spring-base

Apache Zookeeper

https://github.com/4finance/micro-infra-spring


http://zookeeper.apache.org/


Who can I talk to?you just have to:

create a microservice metadata file (i.e. microservice.json)

{

"pl": {

"this": "com/ofg/loans/pl/backoffice-vivus-pl",

"dependencies": {

"bluecash-pl": "com/ofg/payments/pl/bluecash/bluecash-vivus-pl"

}

}

}

src/main/resources/microservice.json


add 4finance micro-infra-spring-base dependency to your project

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-infra-spring-base:0.5.2'

}



add service discovery module

@Configuration

@Import([ServiceDiscoveryConfiguration])

class MyConfiguration {

}

https://github.com/4finance/micro-infra-spring#service-discovery

Microservice metadata file

{

"pl": {


"dependencies": {


}

}

}



{

"pl": {


"dependencies": {


}

}

}

Microservice context (realm)


{

"pl": {


"dependencies": {


}

}

}

name of services I talk with


{

"pl": {


"dependencies": {


}

}

}

paths to services I talk within context (realm)

Where are my collaborators?{

"pl": {


"dependencies": {


}

}

}

COLLABORATOR’S PATH IN ZOOKEEPER

pl/com/ofg/payments/pl/bluecash/bluecash-vivus-pl

Where are my collaborators?

PLpl/com/ofg/loans/payment-processor

pl/com/ofg/loans/risk-analyzer

LTlt/com/ofg/loans/payment-processor

lt/com/ofg/loans/risk-analyzer

How can I contact others?if you want to call your collaborator (e.g. ‘bluecash-pl’) just:

add micro-infra-spring-base dependency to your project

repositories {

jcenter()

}

dependencies {


}

https://github.com/4finance/micro-infra-spring#module-configuration-8


add Service discovery and Abstraction over RestTemplate modules

@Configuration

@Import([ServiceDiscoveryConfiguration, ServiceRestClientConfiguration])


}


https://github.com/4finance/micro-infra-spring#abstraction-over-resttemplate---bound-with-service-discovery


use 4Finance ServiceRestClient bean

How can I contact others?serviceRestClient.forService('bluecash-pl')

.post()

.onUrl('/api/whatever/123')

.body('''{"some":"json"}''')

.withHeaders()

.contentTypeJson()

.andExecuteFor()

.anObject()

.ofType(String)

COLLABORATOR’S ALIAS FROM MICROSERVICE.

JSON


.post()



.withHeaders()

.contentTypeJson()

.andExecuteFor()

.anObject()

.ofType(String)

HTTP METHOD


.post()



.withHeaders()

.contentTypeJson()

.andExecuteFor()

.anObject()

.ofType(String)

URL ON COLLABORATOR TO CALL


.post()



.withHeaders()

.contentTypeJson()

.andExecuteFor()

.anObject()

.ofType(String)

BODY TO SEND


.post()



.withHeaders()

.contentTypeJson()

.andExecuteFor()

.anObject()

.ofType(String)

HEADERS TO SEND


.post()



.withHeaders()

.contentTypeJson()

.andExecuteFor()

.anObject()

.ofType(String)

RETURN A RESPONSE OF A GIVEN TYPE


.post()



.withHeaders()

.contentTypeJson()

.andExecuteFor()

.anObject()

.ofType(String)

How can others contact me?solved by:


Apache Zookeeper

Netflix Exhibitor - UI for Zookeeper





https://github.com/Netflix/exhibitor

https://github.com/Netflix/exhibitor

How can others contact me?you just have to:

create a microservice metadata file (i.e. microservice.json)

{

"pl": {


"dependencies": {


}

}

}




repositories {

jcenter()

}

dependencies {


}



add service discovery module

@Configuration



}


How can others contact me?

{

"pl": {


"dependencies": {


}

}

}


{

"pl": {


"dependencies": {


}

}

}

Microservice context (realm)


{

"pl": {


"dependencies": {


}

}

}

my path in context (realm)

How can others contact me?{

"pl": {


"dependencies": {


}

}

}

MY PATH

pl/com/ofg/loans/pl/backoffice-vivus-pl

How can others contact me?when microservice boots up it registers itself in Zookeeper under specified path

you can register multiple instances under same path

Zookeeper supports load balancing (Round Robin, Random and Sticky)


/pl/com/ofg/loans/pl/backoffice-vivus-pl

instance id (you can have multiple ones)


Data as String:{ "name": "com/ofg/loans/pl/backoffice-vivus-pl", "id": "15628f76-753e-4a25-83de-de132293a58d", "address": "192.168.0.1", "port": 8080, "sslPort": null, "payload": null, "registrationTimeUTC": 1410343209917, "serviceType": "DYNAMIC", "uriSpec": { ... }}

Instance entry in Zookeeper:



Instance entry in Zookeeper:THE PATH

WHERE MICROSERVICE

REGISTERS ITSELF IN GIVEN

REALM



Instance entry in Zookeeper:

MICROSERVICE REGISTERS

ITSELF IN ZOOKEPER

WITH ADDRESS AND PORT

What’s my API?solved by:

4finance micro-infra-spring-swagger

Wordnik Swagger

4finance micro-infra-spring-swagger-ui

Wordnik Swagger-UI



https://github.com/wordnik/swagger-spec



https://github.com/wordnik/swagger-ui

What’s my API?you just have to add dependencies to your project:

micro-infra-spring-swaggermicro-infra-spring-swagger-ui

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-infra-spring-swagger:0.5.2'

compile 'com.ofg:micro-infra-spring-swagger-ui:0.5.2'

}





What’s my API?from micro-infra-spring-swagger:

add Swagger module to Spring’s context

@Configuration

@Import([SwaggerConfiguration])


}


https://github.com/4finance/micro-infra-spring#swagger---api-documentation

thanks to micro-infra-spring-swagger-ui:

swagger-ui will be present under /src/main/resources/static/swagger/

What’s my API?


and now your microservice publishes its API by itself!

What’s my API?

What’s my API?

What’s my API?

RESTCONTROLLERS

What’s my API?

METHOD TYPES

What’s my API?

METHOD URLS METHOD NAMES

What’s my API? - Try me out!

What’s my API? - Try me out!

ENTER PARAMETERS

SEND A REQUEST

How to be part of a bigger system: logs?Question:

How many lines of logs were produced on production by PL Cola during 1 hour?

How to be part of a bigger system: logs?Answer:

73 000 lines of logs

How to be part of a bigger system: logs?

You are not going to grep that, are you?

How to be part of a bigger system: logs?solved by:

logback log pattern


https://github.com/4finance/boot-microservice/blob/master/src/main/resources/logback.groovy





logback log pattern


SO THAT WE HAVE A STANDARDISED WAY OF

LOGGING






logback log pattern

4finance micro-infra-spring-base SO THAT CORRELATION ID IS ALWAYS SET FOR YOU





How to be part of a bigger system: logs?correlation ID:

a header value in a requestin each microservice this header is the same for the same business actionyou can see a flow of messages between microservices in a chronological order


COLA

REQUEST

No correlationId

CorrelationId set to X


RESPONSE

FIRST SERVICE

REQUEST



RESPONSE

ANOTHER SERVICE

REQUEST

CorrelationId set to XCorrelationId

set to X

RESPONSECorrelationId set to X

YET ANOTHER SERVICE

REQUEST

CorrelationId set to XCorrelationId

set to X

RESPONSE

How to be part of a bigger system: logs?you just have to:


repositories {

jcenter()

}

dependencies {


}


How to be part of a bigger system: logs?you just have to:

add CorrelationId module to Spring’s context

@Configuration

@Import([CorrelationIdConfiguration])


}

https://github.com/4finance/micro-infra-spring#correlationid-setting

How to be part of a bigger system: logs?Application logs in a specified format:

"%d{yyyy-MM-dd HH:mm:ss.SSSZ, Europe/Warsaw} |

%-5level |

%X{correlationId} |

%thread |

%logger{1} |

%m%n"


2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |



TIMESTAMP IN ISO8601 FORMAT


2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |



LOGGING LEVEL


2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |



CORRELATION ID


2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |



THREAD NAME


2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |



CLASS FULLY QUALIFIED NAME (SHORTENED)


2014-09-10 14:34:40.448+0200 |

INFO |

538898f2-543f-48a3-ad9e-3d2961b2b73d |

qtp326276239-140 |


Initializing new backoffice applicationLOG MESSAGE


SAME CORRELATION ID


DIFFERENT MICROSERVICES

How to be part of a bigger system: logs?you can see collected logs from all services in kibana

http://www.elasticsearch.org/overview/kibana/

How to be part of a bigger system: logs?with kibana you can:

query your logs

create dashboards with filtered data

collect logs from different services

How do I expose my metrics?solved by:


Dropwizard Metrics



https://dropwizard.github.io/metrics/3.1.0/

https://dropwizard.github.io/metrics/3.1.0/

How do I expose my metrics?you just have to:


repositories {

jcenter()

}

dependencies {


}


How do I expose my metrics?you just have to:

add metrics publishing module to Spring’s context

@Configuration

@Import([MetricsConfiguration])


}

https://github.com/4finance/micro-infra-spring#metrics-publishing

How do I expose my metrics?we have a template of metric path in Graphite:

apps.(env).(country).(service-name).(dot-separated-metric)

apps.test.pl.bluecash-adapter.transfer.requests.balance.count

“Bluecash adapter’s” count of “transfer” type requests

How do I expose my metrics?you just have to provide (as a property):

# Environment to which you deploy application ('test', 'prod', etc.) - default value: ‘test’

metrics.path.environment=test

# Country for which you deploy your app - default value: ‘pl’

metrics.path.country=pl

# Your app name - default value: ‘service-name’

metrics.path.app=bluecash-adapter

How do I expose my metrics?having properties set you just have to update your metrics:

// treat this code more of a pseudocode ;)Counter transferBalanceRequests = metricRegistry.counter(‘transfer.requests.balance’)

void doSomethingUponTransferBalanceRequest() {

// do sth with the request for transfer balance…

// and update the counter metrictransferBalanceRequests.inc()

}

How do I view my metrics?solved by:

Graphite

Grafana - UI for Graphite

JMX - jconsole, jvisualvm, etc.

http://graphite.wikidot.com/installation

http://graphite.wikidot.com/installation

http://grafana.org/

http://grafana.org/

How do I view my metrics?you can view your metrics in Graphite

you can create dashboards in Grafana

your applictaion metrics are exposed via JMX

How do I view my metrics?

Ok, so how do I start?

But, but, but… it sounds so complicated!

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-infra-spring-swagger-ui:0.5.2'

}

Don’t want to add separate modules?@Configuration@Import([ServiceDiscoveryConfiguration])


}

@Configuration

@Import([SwaggerConfiguration])


}

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-infra-spring-swagger:0.5.2'

}

@Configuration

@Import([MetricsConfiguration])


}

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-inf

ra-spring-base:0.5.2'

}

@Configuration



}

Don’t want to add separate modules?

Good! We’ll do that for you!!You just have to:

add ONE dependency - 4finance micro-infra-spring to your project

import ONE configuration - WebAppConfiguration

create ONE file - microservice.json


One dependency - micro-infra-spring

repositories {

jcenter()

}

dependencies {

compile 'com.ofg:micro-infra-spring:0.5.2'

}

One config - WebAppConfiguration@Configuration

@Import(WebAppConfiguration)


}

If you want you can import only needed pieces or override configuration defaults

One file - microservice.json

{

"pl": {

"this": "com/ofg/your/microservice",

"dependencies": {

"collaborator": "com/ofg/microservice/collaborator"

}

}

}


Question:what should I do to start working on a new microservice?


Answer:

Clone a repo


Without UI:https://github.com/4finance/boot-microservice

With UI:https://github.com/4finance/boot-microservice-gui

https://github.com/4finance/boot-microservice

https://github.com/4finance/boot-microservice

https://github.com/4finance/boot-microservice-gui

https://github.com/4finance/boot-microservice-gui

Ok, so how do I start?those repos:

contain completely set up microservice

are written in Groovy

are built with Gradle (set up for 4finance)

are JDK8 compliant

contain an example of a business scenario

Ok, so how do I start?our libs:

have Javadocs

Ok, so how do I start?our libs:

have ReadMe’s

Where are we?what we have managed to achieve:

Microservice setup (correlationId setting, metrics support, rest template abstraction etc.)

CorrelationId setting for Spring Reactor

Where are we?Libs on production in:

COLA-VIVUS

WEB-VIVUS

BLUECASH-ADAPTER

MYMOID-ADAPTER

WEB-ZAPLOCOLA-ZAPLOBROKERS

Where are we?

Microservice setup (correlationId setting, metrics support, rest template abstraction etc.)

CorrelationId setting for Spring Reactor

CorrelationId setting for Camel

NOW

Fix bugs

Consumer driven contracts (4finance StubRunner)

Circuit breaking (Netflix Hystrix)

Store properties in Zookeeper (Netflix Archaius)

Retry policies for RestTemplate

Resolve issues with circular obligatory dependencies

NEXT

Investigate Riemann and Suro as event servers / data pipelines

New templates for other JVM languages

Sky is the limit

FUTURE

https://github.com/4finance/stub-runner

https://github.com/Netflix/Hystrix/

https://github.com/Netflix/Archaius

Want to contribute?

Want to contribute? ISSUES!

3 steps to eternal fame1. Ask any of the existing 4Finance organization members

to add you to that organization

2. Pick an issue, fix it and create a pull request

3. Once merged profit from eternal fame

https://github.com/4finance

Questions?

4finance microservice pilot in poland

Technology