![Page 1: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/1.jpg)
>> Yorgos Saslis | @gsaslis | DevStaff Meetup, Heraklion, Crete
SCALABLE + MAINTAINABLE +
RELIABLE
APPS
BUILDING
12-factors�to�consider...�
![Page 2: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/2.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
12FACTOR.NET
2
![Page 3: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/3.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
CODEBASE
3
![Page 4: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/4.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
I. CODEBASE1 codebase, kept in a Version control system
1 codebase == 1 app. No cheating.
Shared code is in libraries.
Library sources are NOT included in app sources.
One codebase can have many deploys (dev, test, staging, prod)
4
![Page 5: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/5.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
DEPENDENCIES
5
![Page 6: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/6.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|D
ev
Sta
ff M
ee
tup
, He
raklio
n, C
rete
II. DEPENDENCY DECLARATION
never rely on implicit existence of system-wide packages
e.g. curl does *not* exist everywhere your app will run!! (yes, docker, i’m talking to you)
“declare all dependencies, completely and exactly, via a dependency declaration manifest”
completely: don’t leave anything out. (this is not as easy as it sounds)
exactly: make sure to specify versions, or version upgrade policy
optimistic `>=` vs. pessimistic `~>` version constraints (semantic versioning)
6
![Page 7: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/7.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|D
ev
Sta
ff M
ee
tup
, He
raklio
n, C
rete
II. DEPENDENCY ISOLATIONUse some environment that allows you to ensure you are loading what you’ve declared.
And only that!
if your stack doesn’t provide it, just start with as empty a box as you can find
Examples:
Ruby: gemfile + bundle exec
Python: pip + virtualenv
Java: classpath
7
![Page 8: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/8.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
CONFIG
8
![Page 9: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/9.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|D
ev
Sta
ff M
ee
tup
, He
raklio
n, C
rete
III. CONFIGIS: Everything that varies between deploys.
(db url, credentials, hostnames, etc.)
NOT constants in code. (Config varies substantially across deploys, code does not.)
Avoid config files.
All config in env vars.
9
![Page 10: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/10.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
BACKING SERVICES
10
![Page 11: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/11.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
IV. BACKING SERVICESDatabase. Obviously.
Message queue (rabbitmq, SQS, redis, etc.)
SMTP client (postfix)
Cache (memcached, redis)
Metrics gathering (new relic, loggly)
Binary object stores (Amazon S3)
11
![Page 12: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/12.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
IV. BACKING SERVICESMake no distinction between local and third-party services
`localhost` shouldn’t make a difference to `some_fqdn`
Should be easy to swap simply by changing resource handle (URL) and credentials.
e.g. locally hosted mysql / postgresql <==> AWS RDS
Attachable resources: i am connecting to some db, when that is available.
12
![Page 13: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/13.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
| D
ev
Sta
ff M
ee
tup
, He
raklio
n, C
rete
BUILD - RELEASE - RUN
13
![Page 14: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/14.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
| D
ev
Sta
ff M
ee
tup
, He
raklio
n, C
rete
V. BUILD - RELEASE - RUNStrictly separate the below 3 build and run stages:
build: create executable bundle, using some version, fetching all dependencies and compiling (where necessary).
release: add config to bundle, to have something that’s ready to run.
run: (a.k.a. runtime) launch some process(es) that run the release.
14
![Page 15: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/15.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
| D
ev
Sta
ff M
ee
tup
, He
raklio
n, C
rete
V. BUILD - RELEASE - RUNNo changes to code at runtime.
Use deployment tools for release management:
deploy automatically,
but also **rollback** a release.
Releases are immutable.
Yes, this means you are not allowed to manually make changes to production!
Releases should have some unique ID
timestamp, number increment, hash, etc.15
![Page 16: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/16.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
PROCESSES
16
![Page 17: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/17.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
VI. PROCESSESExecute the app as one or more stateless processes.
Twelve-factor processes are stateless and use a shared-nothing architecture.
Data is persisted in stateful backing services, typically a database.
Use memory or local filesystem ONLY as a brief, single-transaction cache.
Don’t expect the filesystem or memory to be the same.
17
![Page 18: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/18.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
PORT BINDING
18
![Page 19: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/19.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|D
ev
Sta
ff M
ee
tup
, He
raklio
n, C
rete
VII. PORT BINDINGExport services via port binding
Self-contained services that listen to a specific port.
webserver library is bundled into the app. Do not rely on server existing in system.
19
![Page 20: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/20.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
CONCURRENCY
20
![Page 21: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/21.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|D
ev
Sta
ff M
ee
tup
, He
raklio
n, C
rete
VIII. CONCURRENCYSeparate Workload type <—> process type.
(e.g. HTTP requests <—> web process, long-running background tasks <—> worker process.
Many workloads, but also many processes of the same workload!
Rely on some process manager or scheduler to handle lifecycle of your processes (don’t daemonize!)
21
![Page 22: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/22.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
DISPOSABILITY
22
![Page 23: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/23.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
IX. DISPOSABILITY Process must start (and STOP!!) fast!
Minimize startup time (a few seconds!)
Shutdown gracefully (on SIGTERM)
Protect your state against sudden death.
23
![Page 24: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/24.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
| D
ev
Sta
ff M
ee
tup
, He
raklio
n, C
rete
DEV / PROD PARITY
24
![Page 25: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/25.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
| D
ev
Sta
ff M
ee
tup
, He
raklio
n, C
rete
X. DEV/PROD PARITY
25
Traditional app Twelve-factor app
Time between deploys Weeks Hours
Code authors vs code deployers Different people Same people
Dev vs production environments Divergent As similar as possible
![Page 26: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/26.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
| D
ev
Sta
ff M
ee
tup
, He
raklio
n, C
rete
X. DEV/PROD PARITY
Keep development, staging, and production as similar as possible
Use same backing services (same type of) between development and production
e.g. using in-memory dbs is an anti-pattern
Design for continuous deployment
26
![Page 27: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/27.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
LOGS
27
![Page 28: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/28.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
XI. LOGSUse logs as event streams
a stream of aggregated, time-ordered events from all running processes and backing services.
Not concerned with routing or storage of output stream.
Do not attempt to write to or manage logfiles (e.g. rotate). Instead, each running process writes its event stream, unbuffered, to stdout.
In production, a log router / forwarder will take care of sending events from log stream to a centralized logging facility for long-term storage.
where… find events (fast!), spot patterns (graphs over time), alerting!
28
![Page 29: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/29.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|
De
vS
taff
Me
etu
p, H
era
klio
n, C
rete
ADMIN PROCESSES
29
![Page 30: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/30.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
|D
ev
Sta
ff M
ee
tup
, He
raklio
n, C
rete
XII. ADMIN PROCESSESone-off tasks:
in an identical environment as long-running processes,
running against a release,
using same codebase and
using same config.
generally shipped together with production code, using same dependency isolation.
they become part of the release30
![Page 31: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/31.jpg)
>> Y
org
os S
aslis
|
@g
sa
slis
| D
ev
Sta
ff M
ee
tup
, He
raklio
n, C
rete
31
![Page 32: Building Horizontally Scalable, Maintainable and Reliable Software](https://reader031.vdocument.in/reader031/viewer/2022030314/58996f1f1a28ab49478b5273/html5/thumbnails/32.jpg)
>> Y
org
os S
aslis
|
So
ftw
are
De
ve
lop
er
| @
gsa
slis
THANK YOU!
32
Yorgos Saslis @gsaslisgithub.com/gsaslis