rise of the machines - automate your development

Rise of the machinesAutomate your development

@svenpet

Disclaimer

Scripts Chef, Puppet, etc better build pipelines

Disclaimer

Ideas through examples

Rise of the machines

ROBOTSare coming to our day to day lives


IN Software Development

Sven PetersAtlassian Evangelist

@svenpetsvenpet.com

Automation in Software Developmentmy personal story

build scripts (make)

1996JUnit

2001

2004

ant -> later maven

2005

Eclipse (code generation)

Hudson

2006

2009

Puppet

1998

GUI builder2013

Docker

Build & TestsDeployments

Small scripts

What are we automating right now?

What are we automating right now?

Code Reviews

Stand Ups

Ops

Support

Merging

Engineering Health

Bots everywhere

Coding bots Ops bots Service bots Doc botsReport botsTest bots

Welcome… orI’m new

Add ons

PostgresScala

IntelliJ

Git

JDK

Happy installing & setup… for the next 8 hours

Agent CharlieMeet

Download

Install

Configure

Agent CharlieMeet

Puppet, Shell & RubyWorks with OSX & Ubuntu

Atlassian Plugin SDK

Bitbucket SSH Keys

Git

Mercurial

HipChat

HomeBrew

IntelliJ IDEA

JDK

Maven

PostgreSQL

Stash Keys

and much more…

Without Agent Charlie

With Agent Charlie

Coding botsAutomate your Coding phase

27’’ is not enough

Chat

EmailRepo

manager

IDE

Command

line

Issue

Tracker

Issue tracker workflow

TODO IN PROGRESS IN REVIEW DONE

BRANCH CODE REFACTOR PULL MERGE

Developer workflow


What you should do



What you do


Keep the team updated!

Smart commitsmention issue_key and #trigger the issue tracker

Github JIRA YouTrack Trello Bitbucket

Control the issue by commits


JRA-566 #start-Progress

JRA-566 #start-review

fixes #566

No tool switching

Robot is just receiving

commands


branchcommit

Controlled by action

pullcreate review

request

mergeclose review

Our workflow


branch pull merge

branch pull merge

master

JRA-554-bugfix

branch pull merge

master

commits code directly to the master branch!

branch pull merge

master

branch pull merge

master

JRA-554-bugfix

bad surprisesh*t happens, but can be avoided

bad surprisesh*t doesn’t happen that often

no

branch pull merge

don’t branch from a red build

master

JRA-554-bugfix

branch pull merge

master

JRA-554-bugfix

invite the best reviewers

Recent comitters or author

auto suggest reviewer

branch pull merge

minimum 2 approvals

master

JRA-554-bugfix

branch pull merge

unapprove reviewers when code changes

master

JRA-554-bugfix

The need to update older versions

master

Version 1.1

Version 1.0

Support older versions

master

Version 1.1

Version 1.0

JRA-234-hotfix


master

Version 1.1

Version 1.0

JRA-234-bugfix


boring!

Huhu I can do it!

master

Version 1.1

Version 1.0

JRA-234-bugfix

auto merges

Test botsSMARTER Test Automation

&:hover { text-decoration: none; color: @invertedBodyTextColor background-color: #366ca6; cursor: pointer;

}

&:hover { text-decoration: none; color: @invertedBodyTextColor background-color: #366ca6; cursor: pointer;

}

Static code analysis can catch this!Findbugs - Checkstyle - PMD

Listen to what Freud tells you about

source codeyour

Freud Bot

&:hover { text-decoration: none; color: @invertedBodyTextColor background-color: #366ca6;

cursor: pointer; }

You should use a variable instead for . I suggest that you use a variable of similar color: #366CA6

@globalHeaderPrimaryActionFallbackColor @globalHeaderTopColor @primaryLinkColor

Freud Botfor Pull Requests

Freud Bot

less review work makes suggestions

We are trying hard to write awesome code

packagecom.miguelcatalan.materialsearchview;

importandroid.app.Activity;

importandroid.widget.ListView;

importandroid.widget.RelativeLayout;

importandroid.widget.TextView;

importjava.lang.reflect.Field;

importjava.util.List;

/**

*@authorMiguelCatalanBañuls

*/

publicclassMaterialSearchViewextendsFrameLayoutimplementsFilter.FilterListener{

privateMenuItemmMenuItem;

privatebooleanmIsSearchOpen=false;

publicMaterialSearchView(Contextcontext){

this(context,null);

}

publicMaterialSearchView(Contextcontext,AttributeSetattrs){

this(context,attrs,0);

@OverridepublicvoidonTextChanged(CharSequences,intstart,intbefore,intcount){

mUserQuery=s;

startFilter(s);

MaterialSearchView.this.onTextChanged(s);

}

}

LOC: 32

MyAwesomeClass

One month later

packagecom.miguelcatalan.materialsearchview;

importandroid.app.Activity;

importandroid.content.Context;

importandroid.content.Intent;

importandroid.content.pm.PackageManager;

importandroid.content.pm.ResolveInfo;

importandroid.content.res.TypedArray;

importandroid.graphics.Rect;

importandroid.graphics.drawable.Drawable;

importandroid.os.Build;

importandroid.os.Parcel;

importandroid.os.Parcelable;

importandroid.speech.RecognizerIntent;

importandroid.text.Editable;

importandroid.text.TextUtils;

importandroid.text.TextWatcher;

importandroid.util.AttributeSet;

importandroid.util.Log;

importandroid.view.KeyEvent;

importandroid.view.LayoutInflater;

importandroid.view.MenuItem;

importandroid.view.View;

importandroid.view.inputmethod.InputMethodManager;

importandroid.widget.AdapterView;

importandroid.widget.EditText;

importandroid.widget.Filter;

importandroid.widget.Filterable;

importandroid.widget.FrameLayout;

importandroid.widget.ImageButton;

importandroid.widget.ListAdapter;

importandroid.widget.ListView;

importandroid.widget.RelativeLayout;

importandroid.widget.TextView;

importcom.miguelcatalan.materialsearchview.utils.AnimationUtil;

…one month later

LOC: counting….

MyAwesomeClass

//RequestFocus

mSearchSrcTextView.setText(null);

mSearchSrcTextView.requestFocus();

if(animate){

AnimationUtil.fadeInView(mSearchLayout,AnimationUtil.ANIMATION_DURATION_MEDIUM,newAnimationUtil.AnimationListener(){

@Override

publicbooleanonAnimationStart(Viewview){

returnfalse;

}

@Override

publicbooleanonAnimationEnd(Viewview){

if(mSearchViewListener!=null){

mSearchViewListener.onSearchViewShown();

}

returnfalse;

}

@Override

publicbooleanonAnimationCancel(Viewview){

returnfalse;

}

});

}else{

mSearchLayout.setVisibility(VISIBLE);

if(mSearchViewListener!=null){

mSearchViewListener.onSearchViewShown();

}

}

mIsSearchOpen=true;

}

/**

*Closesearchview.

*/

publicvoidcloseSearch(){

…one month later

LOC: 986

MyAwesomeClass

Code over time

86%

20

1045

27

test coverage

deprecated methods

old unit tests (JUnit 3)

compiler warnings

Code over time

83% test coverage

23 deprecated methods

1136 old unit tests (JUnit 3)

36 compiler warnings

Code over time

75% test coverage

32 deprecated methods

1289 old unit tests (JUnit 3)

45 compiler warnings

Developers are trying to solve problems

Developers don’t write bad code on purpose

Calling

Dr. Codemeasure constantly the engineering health

measure constantly the engineering health

Calling

Dr. Code

jira.stats.tests.junit.3.count

No more JUnit 3 tests!JUnit 3 JUnit 4

jira.stats.bundled.jars.in.plugins

Inspect all the things

jira.stats.tests.junit.3.count

Hall of Shame!You’ll enter the

JS not minified

API compatibility :(

New JUnit3 test

Caught by Dr Code?

Hall of Shame!You’ll enter the

Wallboard - so everyone sees it!

Hall of Fame!You’ll enter the

Fixed a matrix?

publicabstractclassTask{

publicfinalvoidexecuteWith(Callbackcallback){

execute();

if(callback!=null){

callback.call();

}

}

publicabstractvoidexecute();

}

publicabstractclassTask{

publicfinalvoidexecuteWith(Callbackcallback){

execute();

if(callback!=null){

callback.call(20);

}

}

publicabstractvoidexecute();

}

Old version New version


Hard to spot?

Image Compare Bot

Compare images

(on build)

Alert on differences

(build warning)Easy reporting

(build report)

Our functional tests ran for

h ursso we ran them parallel

Test block 1

Test block 2

Test block 3

Test block 4

Test block 5

statically sized test blocks

waiting for the longest test block

HallelujahFunctional Test Balancer

T1

T 2

T 3

T4

T5

T6

T7

T8

T9

T10

T11

T 12

T13

T14

Hallelujah

build status

Flaky Test Detector

Flaky Test Detector

failing test

rerun test

successful test

test in quarantaine

continue build

create issue

1888 flaky tests detected

817 quarantined tests

fixed

Flaky Test Detector

no log file searching

forces fixing flaky tests

no manual reruns

no more ‘this one again’ moments

OPS botsAutomated Operations

Ops nowproblem

ops

Slow feedback loop

devprob

Faster feedback loop

devprob

Faster feedback loopknows which team -> service responsibility


But that’s a lot of configuration!


add team in deployment descriptor

add connection to

pager duty

Own your codeend-to-end

Prepared for the release?

WAIT…

Release Notes?

New Feature

Improvement

Bugfix

Do you know which features really ship?

Release Branch

Merged JRA-345

Commit messages

+Issue

information

=Release notes

Release Notes Bot

Label in GitHub Issues

Label in GitHub Issues

New features

Improvements

Fixed bugs

Issue Description as

Release NotesHuman readable

issue descriptions

Think about customer value

No surprises

Service botsAutomate your Service

Support Developer

scanning log filesask for log file system knowledge

pause work

Support Developer


Support Hercules


Teaching the robot

Hercules get’s smarter

suggest read

update

{ Well f*%k meTHE ROBOT

was right! }

Your boss has a really important task for your team

Everyone is working from home today

With chat you can reach everyone immediately

All these questions!

How do I download the Sprint report?

How do I set up a build for my

feature branch?

What Java versions do we

support?

!askI’ll look it up for you

Self service for chat

!ask the knowledge base first!

REPORT botsManual Reporting sucks

Stand UpAutomating

s ?

too longsetup VC calls

timezones remind people

Stand UpProblems

Never miss a stand up

Music - Start at 09:29:00

Crontab!

Video - Open at 09:30

No video set up time

Stand up report bots

colocated

distributed

different timezones

Status update

Team update

Stand up bot

Exception

Build failure

New issue

Deployment

Chat Botseverywhere

ΩΩΩΩ

Builds

ΩΩΩΩ

Deployments

ΩΩΩΩ

Exceptions

Chat Botseverywhere

Fast feedback loops Team knowledge Start discussions

The build failed

New deployment

New important

issue

New deployment

New important

issue

New deployment

Get the big picture

Get the big picture

Every team has one or more

http://

atlasb

oard.bitb

ucket.org

/

Every team has one or more

http://

atlasb

oard.bitb

ucket.org

/

see what’s going on point directly

All kind of boards

AdvocatesDeveloper

Support

Bots everywhere

Coding bots Ops bots Service bots Doc botsReport botsTest bots

Reinvent the wheel?

Lots of unmaintained projects

Writing automation code

Very specific problem

No. 1 Problem

10%for better engineering healthwork time

Technical debt

is serious fun!Automation

Go - Ruby - PHP - PythonLanguages

Puppet - Chef - Ansible - DockerTechnologies

Fast feedbackSuccess

Your ProblemsMotivation

From scripts on a developer hard disc

to professional automation

Source ControlAdd code to

Code ReviewsDo

for every change

TestsWrite

Deployment ScriptsHave

Keep your automated systemsunder control

Keep your automated systemsunder control

Lots of different places

Instances for automation services

cost centreow

ner

serv

ice

We use labelsfor better overview

Micros <our own PaaS>

Docker

Java

Go

Python

micros service:create <service-name> Service descriptor

Owner

PagerDuty

Cost Centre

MicrosSimple to create & register

Were are we heading?

Better customer insides

better conclusions

collecting more data


Faster dev feedback loops

less time updating & searching

integrated tools


Better Quality

deeper code analysis

smarter tests


Building software

Parts can be automated

craftis a

Creating customer valueis an

art

Hard to automate


…Skynet became self aware

@svenpetsvenpet.com/talks

I’ll be back…find slides here

Photo

Credits

“VW Golf TDI Clean Diesel was 2010 8983" by Mariordo Mario Roberto Duran Ortiz - Own work. Licensed under CC BY-SA 3.0 via Commons - https://commons.wikimedia.org/wiki/

Backyard Series by syauqee mohamad "http://www.flickr.com/photos/7901820@N07/6945260735"

Tesla Model S by Chrishmt0423 from flickr

Burnt & Abandoned Computer "http://www.flickr.com/photos/99649389@N02/13262802714"