continuous integration with robot sweatshop
TRANSCRIPT
ROBOT SWEATSHOPLightweight, non-opinionated Continuous Integration
WHOAMIgithub.com/JScott
WHATAMIDOINGPlease send help
JENKINS
… <builders> <hudson.tasks.Shell> <command>cd telusmkdir -p targetcucumber -p json_report</command> </hudson.tasks.Shell> </builders>…
– a Jenkins user
“I’d rather work around a bad tool than maintain something new.”
BUILDBOTfrom buildbot.process.factory import BuildFactoryfrom buildbot.steps.source import SVNfrom buildbot.steps.shell import ShellCommand
checkout = SVN(baseURL = '[email protected]:portal-project.git', mode = "update", username = "foo", password = "bar", haltOnFailure = True )
mkdir = ShellCommand(name = "mkdir -p telus/target", command = ["mkdir", "-p", "telus/target"], description = "make the results path")
cucumber = ShellCommand(name = "cucumber -p json_report", command = ["cucumber", "-p", "json_report"], description = "run Cucumber tests")
logresults = ShellCommand(name = "log-cucumber telus/target", command = ["log-cucumber", "telus/target"], description = "upload results")
factory = BuildFactory()factory.addStep(checkout)factory.addStep(mkdir)factory.addStep(cucumber)factory.addStep(logresults)
c['builders'] = [ BuilderConfig(name = "portal-project", slavenames = ['slave1', 'slave2', 'slave3'], factory = factory)]
http://docs.buildbot.net/current/tutorial/fiveminutes.html
“Typically, at first sight it looks like a bunch of complicated concepts that make no sense and whose
relationships with each other are unclear.”
STOP SUFFERING BAD TOOLS
We can do better
ROBOT SWEATSHOP
jobs/portal-project.yaml---branch_whitelist: - develop
commands: - mkdir -p telus/target - cucumber -p json_report - log-cucumber telus/target
SIDE-BY-SIDE CONFIGS<?xml version='1.0' encoding='UTF-8'?><project> <actions/> <description>requires these gems:
statsd-ruby
selenium-webdriver
cucumber
json
RMagick</description> <logRotator class="hudson.tasks.LogRotator"> <daysToKeep>10</daysToKeep> <numToKeep>-1</numToKeep> <artifactDaysToKeep>-1</artifactDaysToKeep> <artifactNumToKeep>-1</artifactNumToKeep> </logRotator> <keepDependencies>false</keepDependencies> <properties/> <scm class="hudson.plugins.git.GitSCM" plugin="[email protected]"> <configVersion>2</configVersion> <userRemoteConfigs> <hudson.plugins.git.UserRemoteConfig> <url>[email protected]:webchannel/uss-cucumber.git</url> <credentialsId>ad5d9dcc-cb2f-42a4-8e9a-f47b3208ce0c</credentialsId> </hudson.plugins.git.UserRemoteConfig> </userRemoteConfigs> <branches> <hudson.plugins.git.BranchSpec> <name>*/develop</name> </hudson.plugins.git.BranchSpec> </branches> <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations> <submoduleCfg class="list"/> <extensions> <hudson.plugins.git.extensions.impl.CloneOption> <shallow>true</shallow> <reference></reference> </hudson.plugins.git.extensions.impl.CloneOption> </extensions> </scm> <canRoam>true</canRoam> <disabled>false</disabled> <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding> <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding> <triggers> <hudson.triggers.TimerTrigger> <spec>H 11 * * *</spec> </hudson.triggers.TimerTrigger> </triggers> <concurrentBuild>false</concurrentBuild> <builders> <hudson.tasks.Shell> <command>cd telusmkdir -p targetcucumber -p json_report -p remote_ff</command> </hudson.tasks.Shell> </builders> <publishers> <org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver plugin="[email protected]"> <testResults>telus/target/features_report.json</testResults> </org.jenkinsci.plugins.cucumber.jsontestsupport.CucumberTestResultArchiver> <hudson.tasks.Mailer plugin="[email protected]"> <recipients>[email protected] [email protected] [email protected] [email protected]</recipients> <dontNotifyEveryUnstableBuild>false</dontNotifyEveryUnstableBuild> <sendToIndividuals>false</sendToIndividuals> </hudson.tasks.Mailer> </publishers> <buildWrapp
from buildbot.process.factory import BuildFactoryfrom buildbot.steps.source import SVNfrom buildbot.steps.shell import ShellCommand
checkout = SVN(baseURL = '[email protected]:portal-project.git', mode = "update", username = "foo", password = "bar", haltOnFailure = True )
mkdir = ShellCommand(name = "mkdir -p telus/target", command = ["mkdir", "-p", "telus/target"], description = "make the results path")
cucumber = ShellCommand(name = "cucumber -p json_report", command = ["cucumber", "-p", "json_report"], description = "run Cucumber tests")
logresults = ShellCommand(name = "log-cucumber telus/target", command = ["log-cucumber", "telus/target"], description = "upload results")
factory = BuildFactory()factory.addStep(checkout)factory.addStep(mkdir)factory.addStep(cucumber)factory.addStep(logresults)
c['builders'] = [ BuilderConfig(name = "portal-project", slavenames = ['slave1', 'slave2', 'slave3'], factory = factory)]
---branch_whitelist: - develop
commands: - mkdir -p telus/target - cucumber -p json_report - log-cucumber telus/target
67
33+
8
USAGE DEMOhttps://asciinema.org/a/22878
http://localhost:10554
THE FUTURERobot Sweatshop
• Output node
• Multiple workers easily
• Implicit job (done!)
• Non-TCP connections
• Worker Docker support
Sweatshop Gears
• Scaffolding scripts
• Custom repos
• Service hub
• Update/uninstall
• List/show
• Docker helper scripts
UNDER THE HOODovercoming some interesting challenges
DISTRIBUTED ARCHITECTURE
github.com/kostya/eye
JOB REQUEST RESILIENCE
GEARS DEMOpackage management tool for non-core bits
https://asciinema.org/a/22878
QUESTIONS?robotsweat.com