Download - Resilient Architecture
![Page 1: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/1.jpg)
RESILIENTARCHITECTUREMattStine( )@mstine
http://www.mattstine.com
![Page 2: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/2.jpg)
HEADLINES
![Page 3: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/3.jpg)
ASYSTEMFAILURECOSTSAWELL-KNOWNRETAILERSIGNIFICANTREVENUEONTHEBIGGESTINTERNETSHOPPINGDAYOF
THEYEAR.
![Page 4: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/4.jpg)
ASYSTEMFAILURECAUSESTHECANCELLATIONOFHUNDREDSOF
FLIGHTS,STRANDINGTHOUSANDSOFAIRLINEPASSENGERS,ANDULTIMATELYCOSTINGTHEAIRLINEMILLIONSIN
REVENUE.
![Page 5: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/5.jpg)
ABEAUTIFULLYDESIGNEDONLINESTORECRUMBLESUNDERTHEPRESSUREOFATHUNDERINGHERDOFCUSTOMERS
TRYINGTOPURCHASETHELATESTTECHGADGET.
![Page 6: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/6.jpg)
ASECURITYBREACHEXPOSESTHOUSANDSOFCUSTOMERCREDITCARDNUMBERS,LEADINGTOMILLIONSINLOSTREVENUEDUETOTHERESULTINGLOSS
OFTRUST.
![Page 7: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/7.jpg)
WHATCANWEDO?
![Page 8: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/8.jpg)
DISRUPTIVECOMPANIESAREALSOAPPROACHINGRESILIENCY
DIFFERENTLY.
![Page 9: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/9.jpg)
STOPTRYINGTOPREVENTMISTAKES.
![Page 10: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/10.jpg)
EMBRACEFAILURE.
![Page 11: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/11.jpg)
FROMMTBFTOMTTR
![Page 12: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/12.jpg)
WENEEDBETTERTOOLSANDTECHNIQUES.
![Page 13: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/13.jpg)
RESILIENTARCHITECTURESEnhanceObservabilityLeverageResiliencyPatternsEmbraceChaos
![Page 14: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/14.jpg)
ENHANCEOBSERVABILITY
![Page 15: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/15.jpg)
SEEFAILUREWHENITHAPPENS
![Page 16: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/16.jpg)
MEASUREEVERYTHING
![Page 17: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/17.jpg)
WHATISNORMAL?ValuesRatesofChangeMean?P95/99/99.9?
![Page 18: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/18.jpg)
WHATISNORMAL?
http://bravenewgeek.com/everything-you-know-about-latency-is-wrong/
![Page 19: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/19.jpg)
![Page 20: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/20.jpg)
![Page 21: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/21.jpg)
SPRINGBOOTHEALTHENDPOINT
{ "diskSpace": { "status": "UP", "total": 1056858112, "free": 878850048, "threshold": 10485760 }, "refreshScope": { "status": "UP" }, "configServer": { "status": "UP", "propertySources": [ "configClient", "https://github.com/spring-cloud-services-samples/fortune-teller/configuration/application.yml" ] }, "hystrix": {
![Page 22: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/22.jpg)
SPRINGBOOTINFOENDPOINT "git": { "build": { "host": "Matts-MacBook-Pro.local", "version": "0.0.1-SNAPSHOT", "time": 1489021333000, "user": { "name": "Matt Stine", "email": "[email protected]" } }, "branch": "master", "commit": { "message": { "short": "initial commit", "full": "initial commit" }, "id": "9b624974e417693cf921b9abc50b5af4ea0b6dde", "id.describe-short": "9b62497-dirty", "id.abbrev": "9b62497", "id.describe": "9b62497-dirty",
![Page 23: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/23.jpg)
DISTRIBUTEDTRACING
Zipkin
![Page 24: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/24.jpg)
EXAMPLES:SpringBootActuatorhttp://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#production-ready
PCFAppsManagerhttps://docs.pivotal.io/pivotalcf/1-9/console/using-actuators.html
SpringCloudSleuthhttps://cloud.spring.io/spring-cloud-sleuth/
Zipkinhttp://zipkin.io/
![Page 25: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/25.jpg)
LEVERAGERESILIENCYPATTERNS
![Page 26: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/26.jpg)
TIMEOUTS
![Page 27: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/27.jpg)
TIMEOUTSThinkingishalfthebattle!AnythingthatblocksthreadsAnymethodcallwithanoptionaltimeoutargument
![Page 28: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/28.jpg)
ADDINGTIMEOUTSTORESTTEMPLATE
@Beanpublic RestTemplate restTemplate() { SimpleClientHttpRequestFactory clientHttpRequestFactory = new SimpleClientHttpRequestFactory(); clientHttpRequestFactory.setConnectTimeout(10 * 1000); // Ten seconds! clientHttpRequestFactory.setReadTimeout(10 * 1000); // Ten seconds! return new RestTemplate(clientHttpRequestFactory);}
![Page 29: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/29.jpg)
RETRIES
![Page 30: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/30.jpg)
RETRIESPotentiallytransientfailuresImmediatelyWithabackoffMaximumtimesLogallthethings
![Page 31: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/31.jpg)
SIMPLERETRY @RequestMapping("/acquireThings")@Retryablepublic ResponseEntity<String> tryToAcquireThings() { logger.info("Attempting to acquire things..."); String things = restTemplate .getForObject("http://localhost:8081/things", String.class); return new ResponseEntity<String>(things, HttpStatus.OK);}
@Recoverpublic ResponseEntity<String> recover() { logger.warn("Returning default response..."); return new ResponseEntity<String>("default things", HttpStatus.OK);}
![Page 32: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/32.jpg)
RETRYWITHBACKOFF @RequestMapping("/acquireThings")@Retryable(maxAttempts = 5, backoff = @Backoff(delay = 100L, maxDelay = 1000L, multiplier = 2, random = true))public ResponseEntity<String> tryToAcquireThings() { logger.info("Attempting to acquire things..."); String things = restTemplate .getForObject("http://localhost:8081/things", String.class); return new ResponseEntity<String>(things, HttpStatus.OK);}
![Page 33: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/33.jpg)
EXPONENTIALBACKOFF @Beanpublic BackOffPolicy backOffPolicy() { return new ExponentialBackOffPolicy();}
![Page 34: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/34.jpg)
BULKHEADS
![Page 35: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/35.jpg)
BULKHEADSMicroservicesThreadPoolsAvailabilityZones
![Page 36: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/36.jpg)
CIRCUITBREAKERS
![Page 37: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/37.jpg)
CIRCUITBREAKERS
![Page 38: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/38.jpg)
SPRINGCLOUDHYSTRIX @HystrixCommand(fallbackMethod = "fallbackFortune")public Fortune randomFortune() { return restTemplate.getForObject("http://fortunes/random", Fortune.class);}
private Fortune fallbackFortune() { return new Fortune(42L, fortuneProperties.getFallbackFortune());}
![Page 39: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/39.jpg)
EXAMPLES:SpringRetryhttps://github.com/spring-projects/spring-retry
Hystrixhttps://github.com/Netflix/Hystrix
viaSpringCloudNetflixhttps://cloud.spring.io/spring-cloud-netflix/
![Page 40: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/40.jpg)
EMBRACECHAOS
![Page 41: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/41.jpg)
HOWDOYOUKNOWYOURSYSTEMWILLTOLERATEFAILURE
IFITHASN'TFAILED?
![Page 42: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/42.jpg)
GAMEDAYEXERCISES
![Page 43: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/43.jpg)
CANWEDIALTHATUPANOTCH?
![Page 44: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/44.jpg)
YAUANDCHEUNG:DESIGNOFSELF-CHECKINGSOFTWARE
(1975)
![Page 45: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/45.jpg)
DIDSOMEBODYSAY...
![Page 46: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/46.jpg)
EXAMPLES:ChaosLemur(BOSH)https://github.com/strepsirrhini-army/chaos-lemur
ChaosLoris(CF)https://github.com/strepsirrhini-army/chaos-loris
![Page 47: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/47.jpg)
![Page 48: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/48.jpg)
REVIEWTIME!StoptryingtopreventmistakesFocusonMTTREnhanceobservabilityLeverageresiliencypatternsEmbracechaos!
![Page 49: Resilient Architecture](https://reader033.vdocument.in/reader033/viewer/2022051520/58ce73481a28abdc578b5f35/html5/thumbnails/49.jpg)
THANKS!
MattStine( )@mstinehttp://www.mattstine.com