understand immutable infrastructure, what? why? how? - meta-meetup devops night
Post on 21-Jan-2018
475 Views
Preview:
TRANSCRIPT
Quentin ADAM from the Clever Cloud
@waxzce on twitter – github- soundcloud – instagram ….
WHO AM I ?
MY DAY TO DAY WORK :
CLEVER CLOUD,
THE IT AUTOMATION COMPANY
KEEP YOUR APPS ONLINE. MADE WITH
NODE.JS, SCALA, JAVA, RUBY, PHP,
PYTHON, GO…
Cloud & on premise ;-)
AND LEARN A LOT OF THINGS ABOUT
YOUR CODE, APPS, AND GOOD/BAD
DESIGN…
LET’S TALK ABOUT
INFRASTRUCTURE
40 times in 5 years price drop of AWS
SERVERS COSTS ARE GOING DOWN
Google I/O 2015
“RACE TO ZERO”
I.E THE SERVER IS FREE
In fact, all you have is…
COOL.
YOU HAVE A SERVER.
HOSTING
Backup Updates Certification
Agreement Process Provisioning
SecurityHardware
maintenance
VALUE
= SERVER
MANAGEMENT
ONCE UPON A TIME, MY BELOVED
SERVERS
THEN, VIRTUALIZED ONES
CREATE HISTORY
DOCUMENTATION
SPECIFIC
TAYLOR MADE
NOT ABLE TO SCALE
KNOWLEDGE
!=
AUTOMATION
Automate
SPLIT COMPLEX PROBLEMS INTO A
COLLECTION OF SMALL ONES
POSSIBLE SERVERS
STATES
…
Updated to V43
New configuration
files
Install new service
History …
AVOID HISTORY
FIX STATE OF
THE DATA TO
ENSURE
ACCESSIBILITY
POSSIBLE STATE IN
AN IMMUTABLE STATE
working
Not working
Very simple to predicate
FIX THE
HISTORY IN A
IMMUTABLE
STATE
BTW,
REAL WORLD
INSIGHT
“WE LIVE IN A MUTABLE WORLD, DEAL WITH IT”
WORLD IS
SUCCESSION
OF
IMMUTABLE
INSTANT
EPHEMERAL
IMMUTABLE
INSTANCES
PET
!=
CATTLE
SERVICE
UPTIME
!=
SERVER
UPTIME
NO MORE OLYMPIC CONTEST ON
UPTIME
And why it’s a problem to manage technical product and decision with book keepers point of view https://www.youtube.com/watch?v=0ip1FoBsLB4
(AWS PRICING HYPOCRISY LEAD TO
BAD PRACTICE)
Summon Yog-Sothoth
HUMANLESS
POWERTOOLS
PRODUCTION HAS TO BE PERFECT
PEOPLE ARE NOT PERFECT
PEOPLE HAVE NOTHING TO DO ON
PRODUCTION STUFF
AT CLEVER CLOUD SSH CONNECTION
IS A RED ALERT
CAPACITY PLANNING
SPLIT PROCESS AND
STORAGE
Storage
• Databases
• Files
• Sessions
• Events
• …
Code
• Can be replicated
• Stateless
• Process
RUNTIME INSTANCES
VM, docker, kubernetes, puppet, ansible, vmware… This is not the problem
TECHNOLOGIES?
Containers, VMs... Comment ces technologies fonctionnent et comment les différencier?
(Quentin Adam)
https://www.youtube.com/watch?v=wG4_JQXvZIc
DOCKER
THERE IS ONE PROCESS:
BUILD
THEN, REGISTER
AND LIVE WITHOUT DATA
AND DIE WHEN MORE
UPDATED VERSION IS THERE
TRASH OLD VERSION
FULLY AUTOMATED HOSTING FACTORY
AN INSTANCE IS DEAD OR ALIVE AND
THAT’S ALL
APP FAIL?
REDEPLOY
NEW
INSTANCE
OPERATION
SYSTEM
UPDATE ?
REDEPLOY
NEW
INSTANCE
REBALANCE
HARDWARE
ALLOCATION?
REDEPLOY
NEW
INSTANCE
PREDICABLE
FAIL OF THE
APPLICATION?
REDEPLOY
NEW
INSTANCE
Picking one instance or another doesn’t matter
STATELESSNESS IS THE KEY
BECAUSE YOU WILL NOT RELY
ON A SERVER ANYMORE
CREATE A FACTORY OF INSTANCE
PROXY
AND
ROUTING
https://github.com/sozu-proxy
https://github.com/sozu-proxy/sozu-
demo/tree/master/kubernetes-using-tube-cheese
https://www.youtube.com/watch?v=y4NdVW9sHtU
https://www.youtube.com/watch?v=Cl_fqWZTYUA
Consul, etcd, Clever Cloud service dependencies
SERVICE DISCOVERY
SIMPLER
WAY,
NEVER
CHANGE ON
RUNTIME
CONFIGURATION
=> ENVIRONMENT VARIABLE
CLEVER CLOUD SERVICE
DEPENDENCIES
https://www.clever-cloud.com/blog/features/2016/06/23/introducing-service-dependencies/
NO OVER
ENGINEERING ON
CONFIGURATION,
WHEN YOU NEED TO
CONFIGURE THE
CONFIGURATION
INJECTOR, YOU ARE
TOO FAR 🖇
HOW TO MANAGE DATA?
SPLIT DATA AND PROCESS:
FS COMPOSITION
Runtime
• local
• replicable
• updated
Data
• file system
• block storage
• replicated
• copy on write?
DATA
STORAGE
WAYS
Not part of this list
DATABASE (ACID OR NOT, SQL OR NOT)
FILE SYSTEM
BLOC STORAGE
OBJECT STORAGE
(COPY ON WRITE)
(COPY ON WRITE)
DATA MANAGEMENT IS BIND
TO DB NEEDS
ACID :
DUPLICATE & DATA EVOLUTION
INSTANCE FOR ONE
ORGANIZATION
ACID
• Atomicity
• Consistency
• Isolation
• Durability
Powerful data
management
• Transaction
• User management
• One above one
Take advantage of ACID
database
BUT IT DESIGNS A
BOTTLENECK
EXAMPLE : E-SHOP ON
CLASSIC MODE
EXAMPLE : E-SHOP ON
CLASSIC MODE
User A buy a
hdd
Database Transaction :
• Stock management
• Order management
• Invoice generation
• Customer Account reward
• …
Transaction
user A is
processed
Stock &
Order are just
perfectly
synchronize
EXAMPLE : E-SHOP ON
CLASSIC MODE
User A buy a
hdd
Database Transaction :
• Stock management
• Order management
• Invoice generation
• Customer Account reward
• …
Transaction
user A is
processed
Stock &
Order are just
perfectly
synchronize
User B buy a
hdd Transaction
user B is
processed
then
EXAMPLE : E-SHOP ON
MULTI-TENANT MODE
i.e. : Multiple shop of various sellers on the same instance
EXAMPLE : E-SHOP ON
MULTI-TENANT MODE
User A buy a
hdd on seller A
Database Transaction :
• Stock management
• Order management
• Invoice generation
• Customer Account reward
• …
Transaction
user A is
processed
Stock &
Order are just
perfectly
synchronize
User B buy a
book on seller B Transaction
user B is
processed
then
HIGH AVAILABILITY:
DISTRIBUTED SYSTEM
DISTRIBUTED
DB AND
HARDWARE
DAFUCKNESS
FORGET THE SAN, GO FULL
DISTRIBUTED
NO SILVER
BULLET
STORAGE
BE CONSISTENT AND PREDICTABLE
MONITORING
IS
MANDATORY
EXPOSE METRICS
BETA ALERT
IMMUTABILITY
AS A PATTERN
ANOTHER TALK FROM ME ;-)
FR version : https://www.youtube.com/watch?v=y1BR7Rh7FSY&t=15s
SECURITY
VOODOO
LIKE
SECURITY IS EVOLVING
https://www.clever-cloud.com/blog/company/2016/04/04/security-is-a-process/
https://www.clever-cloud.com/blog/guests/2015/06/16/the-end-of-the-fortress-metaphor/
FROM SERVER TO
SERVERLESS:
CODE
LOCKED?
F*CK 12
FACTORS?
TCP ?
GO BACK ON
APPLICATION
SERVER?
@waxzceTHANK YOU !
ANY QUESTIONS ?
Clever Cloud gift coupon:
devopsNIGHT17
quentin.adam@clever-cloud.com
https://www.eventbrite.com/e/billets-jenkins-community-day-paris-2017-
33850605071?discount=jcdisclever5774
jcdisclever5774
top related