adrian cockcroft’s architecture

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Adrian Cockcroft’s architecture trends and topics for 2021

Adrian Cockcroft

VP Cloud Architecture Strategy

AWS

A R C 2 1 3

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Serverless first

Chaos engineering and failing over without falling over

Wardley Mapping for technology evolution

Hardware trends

Sustainability

Agenda

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Serverless first

Trends: Microservices

MicroXchg Berlin event where a group of speakers including @adrianco adopted the term

Start with serverless microservices

Definition from a talk by @adrianco in 2014

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.

We chose microservices to get speed and agility . . .

So how long does it really take to build a new application from scratch?

AWS re:InventN O N - P R O F I T H A C K A T H O N F O R S O C I A L G O O D

SkyTruth

Best Friends Animal Society

Several teams

of ~four people

Non-profit organization

with a problem

Monday, 8 AM

Here’s a set of

problems. Pick

one and build a

solution by 8 PM

this evening.

“ “+ =

“ “SkyTruth

Best Friends Animal Society

Several teams

of ~four people

Non-profit organization

with a problem

Monday, 8 AM

Here’s a set of

problems. Pick

one and build a

solution by 8 PM

this evening.+ =

AWS re:InventN O N - P R O F I T H A C K A T H O N F O R S O C I A L G O O D

Where would you start?

Every team used

AWS Lambda 1 Less than

a day total2

Extremely functional

and scalable prototypes4Many team

members’ first time

using AWS Lambda

3

2016 AWS re:Invent HackathonA L I G H T B U L B M O M E N T

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Note: See re:Invent 2019, SVS343

So . . . why doesn’t everyone use serverless first?

Language support

Scalability (new: 10 GB/6 vCPU)

Startup and network latency

Databases / storage interfacing

Security

State handling, event processing

Run duration

Complex configs (new: AWS Proton)

Portability (new: container support) Getting started

See my Serverless-First Function talk for a deep dive https://aws.amazon.com/blogs/compute/icymi-serverless-first-function/

Common concerns

Update S E R V E R L E S S S C A L E

AWS Lambda per-function

maximum disk space

Local

512 MB

Amazon EFS

Petabytes

47.9 TiB per file

June 2020 introduction of support for NFS mounting an

Amazon EFS file system – up to 25,000 concurrent connectionshttps://aws.amazon.com/blogs/compute/using-amazon-efs-for-aws-lambda-in-your-serverless-applications/

UpdateS E R V E R L E S S - F I R S T C O N T A I N E R S

New support for

common build pipelines

delivering containers for

both runtimes

Serverless AWS Fargate

re:Invent 2020 introduction of AWS Lambda support for container packaging

and AWS Proton delivery tooling makes serverless first an even better choice!

Easier transition from spiky

workloads to Always On capacity

Bottom line

Serverless is the fastest way

to build a modern application

If you object, let us know what

we should work on next!

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Chaos engineering and failing over without falling over

Chaos engineering definition(Adrian’s concise version)

Experiment to ensure that the impact of failures is mitigated

Trends – chaos engineering – 10 years on . . .

2010 – First Netflix blog post about Chaos Monkey

Past Present Future

?

Disasterrecovery

Chaosengineering

Continuousresilience

Resilience

We build redundancy into systems so that if

something fails, we can fail over to an alternative

However, our ability to fail over is complex and

hard to test, so often the whole system falls over

How can we do better?

The last strand that breaks is not the cause of a failure!

Build resilient systems like a rope, not a chain, but make sure you know how

much margin you have and how “frayed” your system is

Infrastructure

Switching

Application

People

Chaos

engineering

team

To

ols

Security

red

team

To

ols

Four layers, two teams, an attitude

Experiment to ensure that the effect of failures is mitigated

Chaos architecture

New! AWS Fault Injection Simulator

Run experiments to ensure that both availability and performance impact of failures are mitigated

“You can’t legislateagainst failure; focus on

fast detection and response.”

—Chris Pinkham

Observability Control

Let’s see what we can learn from experts who have been

working on controlling safety critical systems for decades

Engineering a Safer WorldSystems Thinking Applied to Safety – 2012

Professor Nancy G. Leveson – MIT

STPA – Systems Theoretic Process Analysis

STAMP – Systems Theoretic Accident Model & Processes

http://psas.scripts.mit.edu for handbook and talks

STPA model

control diagramUnderstand hazards that

could disrupt successful

application processing

Observabilityand control

Data plane

Throughput

Control plane

Human controller

Model of automation

Model of controlled process

DisplaysControls

Control algorithmModel of

controlled process

Web service

SensorsActuators

Control action generation

Environmental inputs

Written / trained procedures

Completed actions

Customerrequests

Disturbances

Data plane

Throughput

Control plane

Human controller

Model of automation

Model of controlled process

DisplaysControls

Control algorithmModel of

controlled process

Web service

SensorsActuators

Control action generation

Environmental inputs

Written / trained procedures

Completed actions

Customerrequests

Disturbances

What happens

if there is a

big enough

disturbance

to break the

web service?

Large-scale failure

Application is “out of control,” many possibilities

Automation has failed

Network partition, no route connecting application to customers

Application crashed or corrupted, not easily restartable

Anything else you didn’t think of

Trigger failover to alternative system

Large-scale failures

Human controllers

Router control plane

Routing service

Control plane A

Web service A

Availability Zone A

Control plane B

Web service B

Availability Zone B

Control plane C

Web service C

Availability Zone C

Symmetry

and assertions• Services and data

are consistent across

three zones

• Zone failure modes

are independent

• Application should

work normally with

any zone offline

• Routing service

manages failover

ScenarioA W S A V A I L A B I L I T Y Z O N E S

Human controllers

Routing service

Control plane A

Web service A

Availability Zone A

Control plane B

Web service B

Availability Zone B

• Router control plane

detects offline AZ, stops

routing traffic

to it, and retries requests

on the online AZs

• Automated response;

what could go wrong?

Router control plane

ScenarioA W S A V A I L A B I L I T Y Z O N E S

Human controllers

Router control plane

Routing service

Control plane A

Web service A

Availability Zone A

Control plane B

Web service B

Availability Zone B

• Missing updates

• Zeroed

• Overflowed

• Corrupted

• Out of order

• Updates too rapid

• Updates infrequent

• Updates delayed

• Coordination problems

• Degradation over time

STPA hazardsS E N S O R M E T R I C S C H E C K L I S T

Routing control

plane doesn’t clearly

inform humans that

everything is taken

care of, and offline

zone delays and

breaks other metrics

with a flood of errors

Human controllers

Router control plane

Routing service

Control plane A

Web service A

Availability Zone A

Control plane B

Web service B

Availability Zone B

• Model mismatch

• Missing inputs

• Missing updates

• Updates too rapid

• Updates infrequent

• Updates delayed

• Coordination problems

• Degradation over time

STPA hazardsM O D E L P R O B L E M S C H E C K L I S T

Confused human

controllers disagree

among themselves

about whether

they need to do

something or not,

with floods of errors,

displays that lag

reality by several

minutes, and out-of-

date runbooks

Human controllers

Router control plane

Routing service

Control plane A

Web service A

Availability Zone A

Control plane B

Web service B

Availability Zone B

• Not provided

• Unsafe action

• Safe but too early

• Safe but too late

• Wrong sequence

• Stopped too soon

• Applied too long

• Conflicts

• Coordination problems

• Degradation over time

STPA hazardsH U M A N C O N T R O L A C T I O N C H E C K L I S T

Human controllers

should not need

to do anything!

However, they

are confused and

working separately,

trying to fix different

problems – some of

their tools don’t get

used often, and

are broken or

misconfigured to do

the wrong thing

Human controllers

Instead of failing

over, system

falls over

Most likelyresult

In-rush of extra

traffic from failed

zone, and extra work

from a cross zone

request retry storm

causes zones A and B

to struggle and

triggers a complete

failure of the

application –

meanwhile, the

routing service also

has a retry storm

and is impacted

How to fail over without falling over

Alert correlation

Floods of alerts need to be reduced to actionable insights (new: Amazon DevOps Guru)

Observability system needs to cope with floods without failing

Run regular chaos engineering experiments (new: AWS Fault Injection Simulator)

Retry storms – prevent work amplification

Reduce retries to zero except at subsystem entry and exit points

Reduce timeouts to drop orphaned requests

Route calls within the same zone

Symmetries

High level of automation, consistent configuration as code

Consistent instance types, services, versions, zones, and Regions

Principles

If it can be the same, make it look and act identically

If it’s different, make that clearly visible

Test your assumptions continuously

What about multi-Region?

Get AZ failover solid

before attempting

multi-Region

Use STPA to

analyze multi-Region-

specific hazards

Follow

Well-Architected

guide patterns

As data centers migrate to the cloud, fragile and manual disaster recovery processes have been standardized and automated

Testing failure mitigation has moved from a scary annual experience to automated

continuous resilience

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Wardley Maps

Early adopters are using Wardley Maps

Maps always allow you to define position and movement

What is a Wardley Map?

Wardley Maps help define strategic context, barriers, and actions

Examples

• Evolution of technology from custom to utility• Technology stack migrations from data center to cloud• Make vs. buy decisions• Open-source product strategy

Mapping open source for Kubernetes

Too many

projects in the cloud-

native landscape

All in

different stages

of evolution

So let’s try and

put some of them

on a map!

CNCF landscapeH T T P S : / / L A N D S C A P E . C N C F . I O – N O T A M A P , A N E Y E C H A R T !

Projects outside CNCF CNCF projects Services

Mapping value chain against evolutionT H I S I S A M A P ! C L O U D - N A T I V E K U B E R N E T E S C O R E P R O J E C T S

Projects outside CNCF CNCF projects Services

New! AWS Grafana and Prometheus services R E M E M B E R : A L L M A P S A R E W R O N G , B U T S O M E M A P S A R E U S E F U L – U S E T O F A C I L I T A T E D I S C U S S I O N

Results from mapping

Successful products and services due to better:

Shared understanding of moves

Situational awareness

Application of doctrines

Strategic decision-making

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Hardware trends

Data that’s too big to process on one machine

What is “big data”?

But the machines keep getting bigger

EC2 UltraClusters4 , 0 0 0 + A 1 0 0 G P U S O N 4 0 0 G B I T L I N K S I N T O P E T A B I T - S C A L E F A B R I C

Gigabits

bandwidth

10,000

100

10

320 GB 40 GB1,100 GB24,000 GB4,400,000 GB

1,000

1

Memory capacity vs. bandwidth

Engulf your data

in memory to

reduce overhead

Internal memory bandwidth of A100

Terabyte+/second

Network bandwidth per node

comparable to Intel Cascade

Lake main memory bandwidth

EC2 UltraCluster

UltraCluster 4,000+ P4d

400 Gbit Ethernet

Network bandwidth

per node

AWS u24-tb1

8x Intel

Cascade Lake

main memory

Single

NVIDIA A100

1.2 TB/s

GPU memory

AWS P4d

8x NVIDIA A100

NVLINK 600 GB/s

GPU memory

AWS P4d

2x Intel Cascade Lake

main memory

Memory capacity

How can we take advantage of more memory?

In-memory analytics

(e.g., SAP HANA) is

mainstream today

Very large

graph databases(it’s hard to partition graphs)

Use the memory

interconnect as a low-

latency network?

Put all the pods

and instances on

the same node?

Fast replication to

another node, in a

different AZ, for

persistence?

Use shared

memory to avoid

serialization overhead?

Interesting work to watch

Twizzler

UC Santa Cruz / Daniel

Bittman – data-oriented

operating system

Brytlyt

Database that runs

completely inside the GPU

MemVerge

Distributed

memory objects

SQream

GPU accelerated

data warehouse

Zenotech

GPU-based CFD simulation

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Sustainability

The Climate Pledge

Reach the Paris Agreement

BY 2040

Climate Pledge fund

$2 BILLION

Signatories

https://www.theclimatepledge.com

Renewable Energy

https://sustainability.aboutamazon.com/environment/sustainable-operations/renewable-energy

Reducing carbon by moving to AWS

https://sustainability.aboutamazon.com/environment/the-cloud

Amazon Sustainability Data Initiative significantly reduces the

cost, time, and technical barriers

associated with analyzing large datasets to

generate sustainability insights

Data Insights

Information Actionable knowledge

We

ath

er

& c

lim

ate

Re

mo

te s

en

sin

g

CMIP6 climate projections

LOCA downscaled climate projections

Atmospheric Models from Météo-France

NOAA’s Global Surface Summary of Day

HIRLAM Weather Model

NOAA Global Ensemble Forecast System (GEFS)

NOAA Global Forecast System (GFS)

NOAA High-Resolution Rapid Refresh (HRRR) Model

…

JMA Himawari-8

Landsat 8

NOAA GOES 16 & 17

Sentinel-1

MODIS MYD13A1, MOD13A1, MYD11A1, MOD11A1, MCD43A4

Terra Fusion Data Sampler

CBERS

…

Enabling easy access to petabytes of foundational data

Hosted free of charge in public Amazon S3 buckets

Managed and updated by data owners

Amazon Sustainability Data Initiative – ASDI

OS-Climate is an open-source platform to enable access to critical

data and models to support climate risk assessment

https://www.os-climate.org/

Climate risk: OS-Climate

AWS contributions to OS-Climate

https://www.os-climate.org/

CALL TO ACTION

Use AWS services and open data programs to architect solutions and achieve your sustainability goals

Look for opportunities to optimize your architecture

Share the solutions you build to support sustainability goals and collaborate with others in your industry and cross-sector

Resources

Related re:Invent 2020 talks

ARC207 – Architecting resiliency, recovery, and reliability: Capital One

ARC307 – Going global in minutes: Multi-Region expansion simplified (Autodesk)

ARC316 – Testing resiliency using chaos engineering

Related blog posts

“Failing over without falling over”:https://stackoverflow.blog/2020/10/23/adrian-cockcroft-aws-failover-chaos-engineering-fault-tolerance-distaster-recovery

Full Serverless-First talks: https://aws.amazon.com/blogs/compute/icymi-serverless-first-function

Response time variation: https://dev.to/aws/why-are-services-slow-sometimes-mn3

Retries and timeouts: https://dev.to/aws/if-at-first-you-don-t-get-an-answer-3e85

Paper

“Building Mission-Critical Financial Services Applications on AWS,” by Pawan Agnihotri with contributions by Adrian Cockcroft: https://d1.awsstatic.com/Financial%20Services/Resilient%20Applications%20on%20AWS%20for%20Financial%20Services.pdf

Book recommendations

Reading list at: http://a.co/79CGMfB

Adrian Cockcroft@adrianco

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.

AWS Well-Architected Framework

The official best practices for architecting in the AWS Cloud

aws.amazon.com/architecture/well-architected

AWS Well-Architected Labs

Hands-on labs to help you learn, measure, and build using architectural best practices

wellarchitectedlabs.com

AWS Architecture Center

Official AWS repository for all architecture resources

aws.amazon.com/architecture

AWS Solutions Library

Vetted reference implementations and Well-Architected patterns

aws.amazon.com/solutions

Architecture resources

Thank you!

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Adrian Cockcroft

@adrianco

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.