grails & devops: continuous integration and delivery in the cloud

Benoit Hediard AgoraPulse

Grails & DevOps CI & CD in the Cloud

RUNNING A LEAN STARTUP DEVOPS & GRAILS & AWS

Gr8Conf Europe 2014

agora pulse

2

Benoit Hediard @benorama

LEAN STARTUP Part 1

3

DEPLOYMENT PIPELINE WITH GRAILS & AWS

Demo

DEVOPS CULTURE Part 2

#lean #custdev #startup

#agile #devops

#groovylang #grailsfw #aws

BUZZWORD BINGO

About Me

4

A Social Media Management Platform for Brands

Benoit Hédiard Co-founder & CTO

Grails  Plugins  Author:  AWS  SDK,  Facebook  SDK,  CDN  Asset  Pipeline,  Coveralls,  Raven,  SegmentIO…  

Passionate about •  Lean/DevOps •  Cloud/AWS •  Java/Grails •  UX/UI •  Facebook/Twitter Platforms

@benorama

Client Framework

Server Framework

Cloud provider

SaaS services

Our leanstack

5

Elastic Beanstalk

RDS & DynamoDB

SES & SQS

S3 & CloudFront

Route 53

Check http://leanstack.io !

LEAN STARTUP Part 1

6

What is a Lean Startup?

7

A Startup is a search for a repeatable and scalable business model.  The Customer Development process is the way startups quickly iterate and test each element of their business model. Agile Development is the way startups quickly iterate their product as they learn. A Lean Startup is Eric Ries’s description of the intersection of Customer Development, Agile Development and if available, open platforms and open source. – Steve Blank, author of The Startup Owner's Handbook

Feedback Loop

8

Ideas  

BUILD  

Code  

MEASURE  

Data  

LEARN  

BUILD – MEASURE – LEARN The fundamental activity of a startup is to turn ideas into products, measure how customers respond, and then learn whether to pivot or persevere. All successful startup processes should be geared to accelerate that feedback loop. – Eric Ries, author of The Lean Startup

Paul Graham’s Startup Curve

9

Minimum Viable Product Product Market Fit

TROUGH OF SORROW

J

L From search mode

Move fast, break things To execution mode

Grow fast, scale things

Scale Initial enthusiasm

Reality sets in

Experimenting & pivoting

Hockey stick

Starts working

Minimum Viable Product

10

Manual Pipeline

✔ Collaboration ✗ Manual Build ✗ Manual Testing

✗ Manual Delivery ✗ Manual Provisioning ✗ Manual Deployment ✗ Manual Monitoring

Demo – Part 1

11

Code   Build   Test   Release   Deploy   Monitor  

Code repo Team communication

Agile Project Management

EC2

Hosting

Idea Customer

Semi-automated Pipeline

✔ Collaboration ✔ Continuous Build ✔ Continuous Testing

✔ Continuous Delivery ✗ Manual Provisioning ✗ Manual Deployment ✗ Manual Monitoring

Demo – Part 2

12

Code   Build   Test   Release   Deploy   Monitor  

Continuous integration and delivery

Idea Customer

Test coverage history & stats

Coveralls Plugin

DEVOPS CULTURE Part 2

13

What is DevOps?

14

DevOps is a software development method that stresses communication, collaboration and integration between software developers and information technology (IT) operations professionals.

✔ Collaboration

✔ Automation & Infrastructure as Code ✔ Continuous Integration / Testing ✔ Continuous Delivery / Deployment ✔ Continuous Monitoring

GOALS: reduce friction and increase velocity

Pre-2000’s Developer…

15

The Game of Thrones era Developers

Requirements  

Design  

Development  

Test  

Release  

Maintenance  

Pre-2000’s Waterfall Development

16

Dev  

Test  /  QA  

Ops  

Business  

Idea

Customer

Silo

Silo

Silo

Friction

Friction

Friction

Cycle time: Months or Years

2000’s Developer…

17

The Rocky Balboa era

Ops Dev

2000’s Iterative Agile Development

18

Develop  

Test  Release  

Design  

Dev  

Ops  

Business  

Idea

Customer

Cycle time: Weeks

or Months

Stability

Change

2010’s Developer…

19

The X-Men era Business Dev Ops

One team approach

2010’s Lean & DevOps

20

Develop  

Test  Release  

Design  

Dev  

Ops  

Business  

Idea

Customer

Cycle time: Hours

or Days DevOps

Automated Pipeline Demo – Part 3

21

Code   Build   Test   Release   Deploy   Monitor  

✔ Collaboration ✔ Continuous Build ✔ Continuous Testing

✔ Continuous Delivery ✔ Automated Provisioning ✔ Automated Deployment ✗ Manual Monitoring

Elastic Beanstalk

S3 & CloudFront

RDS

PaaS CDN DB

Managed Services

Idea Customer

Elastic Beanstalk Plugin CDN Asset Pipeline Plugin

Git Branching Model

22

develop beta master (PROD)

Snapshot merge deploy  Elastic

Beanstalk BETA

feature

Feature merge

test  

test  

test  

Snapshot merge Release merge Version upgrade deploy  

Elastic Beanstalk

PROD

test  

test  

test  

test  

test  

test  

test  

Cultural shift

23

Developers must own application-layer code wherever it lives, while ops must own the infrastructure wherever that is. – Donnie Berkholz, analyst at RedMonk

Dev/Test Production

Dev   Ops  ApplicaMons   ApplicaMons  

Infrastructure   Infrastructure  

Dev/Test Production  

Dev   ApplicaMons   ApplicaMons  

Ops   Infrastructure   Infrastructure  

Responsability

Responsability

Confuse of Dev or Ops? Simple rule: if you are praise for Web site success, you are Dev; if you are blame when Web site down, you are Ops – DevOps Borat

Cultural shift

IT Automation + IaaS Enterprise DevOps

24

3. Deploy your App

1. Describe your “Infrastructure as Code”

2. Build and manage your “Automated Infrastructure”

PROs Portability

Control Flexibility

CONs Complexity

Maintenance Amazon

EC2

Google Compute Engine

IaaS (Infrastructure as a Service)

Configuration Management Tools

Portable Containers

Lean DevOps / NoOps

25

PaaS + SaaS 1. Deploy your App and use Managed Services

Cloud SQL Amazon RDS

Relational DB

Cloud Datastore DynamoDB

NoSQL

Cloud Storage Amazon S3

Storage

+ Cloud EndPoints + Elasticache, AppStream, SWF, SQS, SES, FPS, Elastic Transcoder…

Amazon RedShift

Big Data BigQuery

Google App Engine

PaaS (Platform as a Service)

Cloud DNS

Elastic Beanstalk

PaaS (Platform as a Service)

Amazon Route 53

PROs Focus on building applications instead of managing Infrastructure

Automated Pipeline Demo – Part 4

26

Code   Build   Test   Release   Deploy   Monitor  

✔ Collaboration ✔ Continuous Build ✔ Continuous Testing

✔ Continuous Delivery ✔ Automated Provisioning ✔ Automated Deployment ✔ Continuous Monitoring

Application and server monitoring

Business monitoring

Exception monitoring

Idea Customer

Sentry Plugin NewRelic Plugin SegmentIO Plugin

Demo – Part 5

27

Zero Downtime Deployment (blue/green)

40 minutes ago…

28

MVP running on locale dev machine (manual build pipeline)

2 Travis files & few Grails Plugins later…

29

“Unlimited” Horizontal Auto-Scaling (hundred of web servers)

Push-Button Scalable Database with Multi-AZ (up to 32vCPU and 244Go of RAM)

100% Resilient (no single point of failure)

Deployment in < 10 minutes (deployment pipeline)

100% Cloud-based (fully managed/no SSH)

CONCLUSION

30

Conclusion

31

Such an exciting time to be a developer AND an entrepreneur today…

Embrace Lean Embrace DevOps

Become an entrepreneur!

#lean #custdev

#agile #devops

#groovylang #grailsfw #aws

#PaaS #SaaS

LEAN STARTUP Part 1

32

DEPLOYMENT PIPELINE WITH GRAILS & AWS

Demo

DEVOPS CULTURE Part 2

#lean #custdev #startup

#agile #devops

#groovylang #grailsfw #aws

BUZZWORD BINGO

About Me

33

A Social Media Management Platform for Brands

Benoit Hédiard Co-founder & CTO

Grails  Plugins  Author:  AWS  SDK,  Facebook  SDK,  CDN  Asset  Pipeline,  Coveralls,  Sentry,  SegmentIO…  

Passionate about •  Lean/DevOps •  Cloud/AWS •  Java/Grails •  UX/UI •  Facebook/Twitter Platforms

@benorama

Client Framework

Server Framework

Cloud provider

SaaS services

Our leanstack

34

Elastic Beanstalk

RDS & DynamoDB

SES & SQS

S3 & CloudFront

Route 53

Check out http://leanstack.io !

LEAN STARTUP Part 1

35

What is a Lean Startup?

36

A Startup is a search for a repeatable and scalable business model.  The Customer Development process is the way startups quickly iterate and test each element of their business model. Agile Development is the way startups quickly iterate their product as they learn. A Lean Startup is Eric Ries’s description of the intersection of Customer Development, Agile Development and if available, open platforms and open source. – Steve Blank, author of The Startup Owner's Manual

Iridium woes (Fail fast)

•  Conceived in 1987 by Motorala •  Bought a fleet of 15 rockets from Russia,

the U.S. and China •  Built and launch 72 satellites •  Nine months after the first call was made

in 1998, Iridium was in Chapter 11 bankruptcy.

37

5.2 billion dollar business plan mistake

No Business Plan Survives First Contact With A Customer – Steve Blank, author of The Startup Owner's Handbook

Feedback Loop

38

Ideas  

BUILD  

Code  

MEASURE  

Data  

LEARN  

BUILD – MEASURE – LEARN The fundamental activity of a startup is to turn ideas into products, measure how customers respond, and then learn whether to pivot or persevere. All successful startup processes should be geared to accelerate that feedback loop. – Eric Ries, author of The Lean Startup

Paul Graham’s Startup Curve

39

Minimum Viable Product Product Market Fit

TROUGH OF SORROW

J

L From search mode

Move fast, break things To execution mode

Grow fast, scale things

Scale Initial enthusiasm

Reality sets in

Experimenting & pivoting

Hockey stick

Starts working

Minimum Viable Product

40

Manual Pipeline

✔ Collaboration ✗ Manual Build ✗ Manual Testing

✗ Manual Delivery ✗ Manual Provisioning ✗ Manual Deployment ✗ Manual Monitoring

Demo – Part 1

41

Code   Build   Test   Release   Deploy   Monitor  

Code repo Team communication

Agile Project Management

EC2

Hosting

Idea Customer

Semi-automated Pipeline

✔ Collaboration ✔ Continuous Build ✔ Continuous Testing

✔ Continuous Delivery ✗ Manual Provisioning ✗ Manual Deployment ✗ Manual Monitoring

Demo – Part 2

42

Code   Build   Test   Release   Deploy   Monitor  

Continuous integration and delivery

Idea Customer

Test coverage history & stats

Coveralls Plugin

DEVOPS CULTURE Part 2

43

What is DevOps?

44

DevOps is a software development method that stresses communication, collaboration and integration between software developers and information technology (IT) operations professionals.

✔ Collaboration

✔ Automation & Infrastructure as Code ✔ Continuous Integration / Testing ✔ Continuous Delivery / Deployment ✔ Continuous Monitoring

GOALS: reduce friction and increase velocity

Pre-2000’s Developer…

45

The Game of Thrones era Developers

Requirements  

Design  

Development  

Test  

Release  

Maintenance  

Pre-2000’s Waterfall Development

46

Dev  

Test  /  QA  

Ops  

Business  

Idea

Customer

Silo

Silo

Silo

Friction

Friction

Friction

Cycle time: Months or Years

2000’s Developer…

47

The Rocky Balboa era

Ops Dev

2000’s Iterative Agile Development

48

Develop  

Test  Release  

Design  

Dev  

Ops  

Business  

Idea

Customer

Cycle time: Weeks

or Months

Stability

Change

Agile

2010’s Developer…

49

The X-Men era Business Dev Ops

One team approach

2010’s Lean & DevOps

50

Develop  

Test  Release  

Design  

Dev  

Ops  

Business  

Idea

Customer

Cycle time: Hours

or Days DevOps

Lean

Automated Pipeline Demo – Part 3

51

Code   Build   Test   Release   Deploy   Monitor  

✔ Collaboration ✔ Continuous Build ✔ Continuous Testing

✔ Continuous Delivery ✔ Automated Provisioning ✔ Automated Deployment ✗ Manual Monitoring

Elastic Beanstalk

S3 & CloudFront

RDS

PaaS CDN DB

Managed Services

Idea Customer

Elastic Beanstalk Plugin CDN Asset Pipeline Plugin

Git Branching Model

52

develop beta master (PROD)

Snapshot merge deploy  Elastic

Beanstalk BETA

feature

Feature merge

test  

test  

test  

Snapshot merge Release merge Version upgrade deploy  

Elastic Beanstalk

PROD

test  

test  

test  

test  

test  

test  

test  

Cultural shift

53

Developers must own application-layer code wherever it lives, while ops must own the infrastructure wherever that is. – Donnie Berkholz, analyst at RedMonk

Dev/Test Production

Dev   Ops  ApplicaMons   ApplicaMons  

Infrastructure   Infrastructure  

Dev/Test Production  

Dev   ApplicaMons   ApplicaMons  

Ops   Infrastructure   Infrastructure  

Responsability

Responsability

Confuse of Dev or Ops? Simple rule: if you are praise for Web site success, you are Dev; if you are blame when Web site down, you are Ops – DevOps Borat

Cultural shift

IT Automation + IaaS Enterprise DevOps

54

3. Deploy your App

1. Describe your “Infrastructure as Code”

2. Build and manage your “Automated Infrastructure”

PROs Portability

Control Flexibility

CONs Complexity

Maintenance Amazon

EC2

Google Compute Engine

IaaS (Infrastructure as a Service)

Configuration Management Tools

Portable Containers

Lean DevOps / NoOps

55

PaaS + SaaS 1. Deploy your App and use Managed Services

Cloud SQL Amazon RDS

Relational DB

Cloud Datastore DynamoDB

NoSQL

Cloud Storage Amazon S3

Storage

+ Cloud EndPoints + Elasticache, AppStream, SWF, SQS, SES, FPS, Elastic Transcoder…

Amazon RedShift

Big Data BigQuery

Google App Engine

PaaS (Platform as a Service)

Cloud DNS

Elastic Beanstalk

PaaS (Platform as a Service)

Amazon Route 53

PROs Focus on building applications instead of managing Infrastructure

Automated Pipeline Demo – Part 4

56

Code   Build   Test   Release   Deploy   Monitor  

✔ Collaboration ✔ Continuous Build ✔ Continuous Testing

✔ Continuous Delivery ✔ Automated Provisioning ✔ Automated Deployment ✔ Continuous Monitoring

Application and server monitoring

Business monitoring

Exception monitoring

Idea Customer

Sentry Plugin NewRelic Plugin SegmentIO Plugin

Demo – Part 5

57

Zero Downtime Deployment (blue/green)

40 minutes ago…

58

MVP running on locale dev machine (manual build pipeline)

2 Travis files & few Grails Plugins later…

59

“Unlimited” Horizontal Auto-Scaling (hundred of web servers)

Push-Button Scalable Database with Multi-AZ (up to 32vCPU and 244Go of RAM)

100% Resilient (no single point of failure)

Deployment in < 10 minutes (deployment pipeline)

100% Cloud-based (fully managed/no SSH)

CONCLUSION

60

Conclusion

61

Such an exciting time to be a developer AND an entrepreneur today…

Embrace Lean Embrace DevOps

Become an entrepreneur!

#lean #custdev

#agile #devops

#groovylang #grailsfw #aws

#PaaS #SaaS

Reading recommendations

62