business process automation with codebeamer & groovy (updated)
Post on 15-Nov-2014
123 Views
Preview:
DESCRIPTION
TRANSCRIPT
Groovy Business Process
AutomationO. David
Colorado State UniversityDept. of Civil Engineering, Dept. of Computer Science
US Department of Agriculture
Olaf David, CSU/USDA 2
Automate and integrate software project management practices using CB with respect to◦ Trackers◦ Forums◦ Builds◦ Documents◦ Users
by using an efficient scripting approach.
Objective
Olaf David, CSU/USDA 3
A First Exampleimport cbscript.CBimport static cbscript.CB.*import org.codehaus.groovy.runtime.TimeCategory
jf = CB.login("http://localhost:8080", “bond", “pass")proj = jf.projects.find{it.name == "MyProj"}
use(TimeCategory) { needsBuild = proj.trackers.find{it.name ==“Bugs"}.issues.any{ bug -> bug.status == "Fixed" && bug.submitted < 30.days.ago && bug.priority > LOW && bug.hasCommits }
if (needsBuild) { println “Starting build ....“ log = proj.builds.find{it.name == “BuildApp"}.invoke() println " Build done: ${log.status} “ println " Output: ${log.stdOut} " }}
jf.logout();
Olaf David, CSU/USDA 4
Principal Architecture
cbscript
cb-api CB Server
DB SCM
Java API- Low level, session centric, stateless- For a wide range of applications
Groovy API- High level, object centric, and productivity oriented- For process automation
Docs
Olaf David, CSU/USDA 5
Semantic density and flexibility of scripts◦ Usability/Efficiency aspects of scripts vs. plain Java
vs. Web UI Remote Management of (multiple) CB server Leveraging general scripting productivity
(Groovy)◦ Closures, Properties, Duck typing◦ Builder (DSL)◦ Intuitive Date management/use
CB web UI does not allow complex queries No automation using the CB Web UI
Why Scripting?
Olaf David, CSU/USDA 6
Java 1.6+ Groovy 1.6+ cbscript-5.3.1.jar
◦ Contains cb-api.jar of the same CB version CB 5.3.x
◦ Network access to context ‘<cburl>/cb/remote-api’
◦ Project user access (account)
Prerequisites
Olaf David, CSU/USDA 7
API respects CB licenses, access permissions
Login/Logoutimport cbscript.CBimport static cbscript.CB.*
jf = CB.login("http://localhost:8080", “bond", "007")// orjf = CB.login( url: "http://10.177.19.63:8080", login: "bond", password:“pass")
// do some work.
jf.logout();
Olaf David, CSU/USDA 8
meta information can be obtained about the session.
read/only
Session informationassert jf.login == "bond"assert jf.url == "http://localhost:8080"assert jf.user.realName == "Default System Administrator"
Olaf David, CSU/USDA 9
‘projects’ returns a list of projects Using closures to iterate, filter, search,…
Getting Projects// list all projectsjf.projects.each { println " id: ${it.id} name : ${it.name}"}
// find project(s)assert jf.projects.find{it.name == "Moose"}.id == 2374assert jf.projects.find{it.id == 2374}.name == "Moose“
// directjf.project(‘Moose’)
Olaf David, CSU/USDA 10
‘trackers’ returns a list of tracker Use closures to iterate, filter, search
Managing Tracker// get all tracker of the ‘oms’ project.jf.projects.find{it.name == "oms"}.trackers.each { tracker -> print tracker.name print tracker.description }
Olaf David, CSU/USDA 11
Closures help traversing resources Obtain the Bug tracker in the oms project.
Then find all the ‘Closed’ issues
Tracker Issues query// Find issues that are not Closeddef bugTracker = jf.projects.find{ it.name == "oms“ }.trackers.find {it.name == "Bugs"}
bugTracker.issues.findAll{ it.status != 'Closed'}.each{ println it.name}
Olaf David, CSU/USDA 12
Find all issues assigned to me and submitted last week
Tracker Issues query (cont.)bugTracker.issues.findAll{ it.assignedTo == jf.user && it.status == 'New'}.each { println it.name}
Olaf David, CSU/USDA 13
Find a tracker issue that has the string 'big problem' in summary
Tracker Issues query (cont.)bugTracker.issues.findAll{ it.summary.contains('big problem')}.each{ println it.name}
Olaf David, CSU/USDA 14
All 'Resolved' issues assigned to user 'bond' that were submitted the last 10 days
Groovy date use is intuitive!
Tracker Issues query (cont.)...bugTracker.issues.findAll{ it.status == "Resolved" && it.assignedTo.name == "bond" && it.submitted < 10.days.ago}.each{ println it.name}
Olaf David, CSU/USDA 15
All tracker issues submitted by me and do not have an attached SCM commit.
Tracker Issues query (cont.)bugTracker.issues.findAll{ return it.submitter == jf.user && !it.hasCommits}.each{ println it.name}
Olaf David, CSU/USDA 16
All tracker issues with high priority that are still open and are not assigned yet to a developer.
Tracker Issues query (cont.)bugTracker.issues.findAll{ bug -> bug.priority == HIGH && bug.status != “Closed” && bug.assigned == null}.each{ println it.name}
Olaf David, CSU/USDA 17
Use properties ‘summary’, ‘text’, ‘priority’… Priority: LOWEST, LOW, NORMAL, HIGH,
HIGHEST
Submit a new Issue// get the bug trackerdef bugs = jf.projects.find{ it.name == “oms"}.trackers.find {it.name == "Bugs"}
bugs.submit(summary:"My Bug", text:"This is a new Bug", priority:NORMAL)
Olaf David, CSU/USDA 18
Comment on a existing issue.
Comment on an Issue
bug = bugs.submit(summary:"My Bug", text:"This is a new Bug", priority:NORMAL)
bug.comment(text:”please specify”, format:PLAIN_TEXT)
Olaf David, CSU/USDA 19
Upload and attach a file to an issue. (file is not stored in Documents)
Attach a file to an Issue
bug = bugTracker.issues.find{ bug -> bug.priority == HIGH}
bug.attach(file:”/tmp/spec-1.txt”)
Olaf David, CSU/USDA 20
‘forums’ contains all project forums Check for new posts today.
Forums
def newsForum= jf.projects.find{ it.name == "oms"}.forums.find {it.name == "News"}
newsForum.findAll{ post -> post.submitted > 8.hours.ago}.each{ print it.text}
Olaf David, CSU/USDA 21
‘forums’ contains all project forums Check for new posts today.
Post a new Forum message
news= jf.projects.find{it.name == "oms"}.forums.find { it.name == "News“}
news.post(subject:"New message", text:"Content here ...")
Olaf David, CSU/USDA 22
Check for all new posts from “Joe Poster” submitted this week and print out the message text.
Check for Posts
news.findAll{ post -> post.submitter.realName == “Joe Poster” && post.submitted < 1.week.ago}.each{ print it.text}
Olaf David, CSU/USDA 23
Use ‘artifact’ to reference a document or folder in the project ‘Documents’
Always absolute to ‘Documents’ root.
Documents
prj = jf.projects.find{it.name == "cbtest"}doc = prj.artifact(‘download/EC2TW.txt’)
assert doc.bytes.size() == 709assert doc.name == ‘EC2TW.txt’assert doc.directory == false
Olaf David, CSU/USDA 24
Copy a file from a CB server to the local FS
Documents (cont.)
prj = jf.projects.find{it.name == "cbtest"}doc = prj.artifact("download/EC2TW.txt")
new File('c:/tmp/test1').withOutputStream { it.write doc.bytes}
Olaf David, CSU/USDA 25
‘builds’ contains all the builds of a project. Call ‘invoke’ to execute the build on a CB
server. (Invoke is a synchronous call)
Builds build = prj.builds.find{it.name == "build-lib"} log = build.invoke();
Olaf David, CSU/USDA 26
Build ‘log’ captures the result of the build.
Builds (cont.)build = prj.builds.find{it.name == "build-lib"}log = build.invoke();
assert log.output != nullassert log.error == nullassert log.status == “Successful”assert log.successful == trueassert log.startedBy == jf.user
Olaf David, CSU/USDA 27
Create associations by calling ‘relate’ on tasks, artifacts, documents, etc.
Pass in the ‘other’ task, artifact, document and the kind of relationship (RELATED, CHILD, PARENT, DEPENDS)
Associations
bug = bugTracker.submit(summary:"My Bug", text:"This is a new Bug", priority:HIGH)
def lib = prj.artifact("builds/ngmf.jar")
bug.relate(lib, RELATED)
Olaf David, CSU/USDA 28
Create a Association to many issues in one ‘bulk’ operation (use closure)
‘milestone’ is a tracker issue
Associations (cont.)
readyToBuild = prj.trackers.find { it.name == "Bug“}.issues.findAll { it.status == "Fixed" && thisMonth(it.submitted) }
readyToBuild.each { bug -> milestone.relate(bug, DEPENDS) }
Olaf David, CSU/USDA 29
Use ‘CB.link()’ to create interwiki links for task messages, forum posts, artifact descriptions to reference other resources
InterWiki linksbuild = prj.builds.find{it.name == "build-lib"}result = build.invoke();
String wikitext = “””!!Milestone info* Output Log Reference: ${CB.link(result)}“””
// resolves to:// “ Output Log Reference : [BUILDLOG:23456]”
Olaf David, CSU/USDA 30
Direct access to a project/forum/tracker/issue/post/build/.. using its ‘Id’ or ‘name’ (projects only)
No traversing all higher level resources necessary -> very efficient, but you need to know the ID!!!
Resource Access Efficiency vs. Query Flexibility
Efficient and Direct Accessprj = jf.project(‘moose’) // get a project by nameprj = jf.project(1234) // get a project by If
tra = jf.tracker(2345) // get a tracker by idiss = jf.issue(23467) // get a issue by idbld = jf.build(23411) // get a build by id...
Olaf David, CSU/USDA 31
Examples*Problem elevationSCRUM supportSCM commit controlAuditable Build ManagementTracker Status Statistics
* The examples demonstrate the support of some aspects of the topics above
Olaf David, CSU/USDA 32
Objective◦ Increase the priority of issues, based on some
dynamic rules
Benefit◦ Early problem detection, identification, and
communication
SCRUM support
Olaf David, CSU/USDA 33
Increase the priority of all verified bugs submitted before April that are assigned to Joe that he did not fixed… so far.
Call this above every day (e.g. cron)
Simple Problem Elevationbugs.findAll{ bug -> bug.status == “Verified" && bug.assignedTo.name == “Joe” && bug.spendHours == 0 && !bug.hasCommits && bug.submittedAt < new Date(2009,4,1)}.each{ it.priority++}
Olaf David, CSU/USDA 34
Examples*Problem elevationSCRUM supportSCM commit controlAuditable Build ManagementTracker Status Statistics
* The examples demonstrate the support of some aspects of the topics above
Olaf David, CSU/USDA 35
Objective◦ Support some aspects of SCRUM
Benefit◦ SCRUM coverage◦ Sprint meeting preparation
SCRUM support
Olaf David, CSU/USDA 36
Prepare a Sprint review meeting◦ Check which tasks in 'Feature' requests in the
'oms' project were finished over the last 30 days.
SCRUM Support
jf.projects.find{it.name == "oms"}.trackers.find{ it.name == "Features"}.issues.findAll { task -> task.status == "Closed" && task.modifiedAt > 30.days.ago}.each { println "${it.summary} - ${it.description}"}
Olaf David, CSU/USDA 37
Prepare a Sprint review meeting◦ Get all the bugs that are still not closed and were
submitted since April 1st 2008 (last sprint meeting)
SCRUM support (cont.)
jf.projects.find{it.name == "oms"}.trackers.find {it.name == "Bugs"}.issues.findAll{ it.status != "Closed" && it.submittedAt > new Date(2008, 1, 4)}.each { println "${it.summary} - ${it.description}"}
Olaf David, CSU/USDA 38
Creating a sprint backlog for all closed tasks in the UI category of new features.◦ The map contains the last seven days,
key: day diff to today (-1 is yesterday) value: the number of closed tasks
SCRUM Support (cont.)def issues = jf.projects.find{it.name=="oms"}.trackers.find{ it.name == "Features"}.issues.findAll { it.category == ‘UI’}(-7..0).each { day -> map[day] = issues.findAll { it.status == "Closed" && it.modifiedAt < 8.hours.ago }.size()}
println map// [-7:10, -6:3, -5:12, -4:4, -3:10, -2:5, -1:1, 0:2]
Olaf David, CSU/USDA 39
Creating daily burndown chart data ◦ Call this script every day to track the hours
being ‘burned’ in developing new features .◦ Create the graph based on this file.
SCRUM Support (cont.)hours = 0def fissues = jf.projects.find{it.name=="oms"}.trackers.find{ it.name == "Features“}.issues.findAll { it.status == “InDevelopment”}.each { hours += it.estimatedHours – it.spentHours}
f = new File(“burndown.dat”)today = new Date()f.append("${today} , ${hours}")
Olaf David, CSU/USDA 40
Examples*Problem elevationSCRUM supportSCM commit controlAuditable Build ManagementTracker Status Statistics
* The examples demonstrate the support of some aspects of the topics above
Olaf David, CSU/USDA 41
Objective◦ A commit contains additional (meta) information
that triggers process actions such as an issue status change
Benefit◦ This developer’s shortcut saves time
General Pattern for triggering of process actions by commit message content
SCM Commit triggers status change
Olaf David, CSU/USDA 42
A SVN commit message such as
svn commit –m “#1234 (status=Fixed) Changed loop order” will trigger a status change on a task to fixed; no separate UI
operation needed. (Put this in cron and execute frequently)
SCM Commit triggers status change
jf.projects.find{it.name == "OMSProject"}.trackers.find { it.name == "Bugs“}.issues.findAll{ it.hasCommits && it.status != "Closed“}.each { task -> task.commits.each { if (it.message ==~ /.*\(status=(.*)\).*/ && today(it.commitedAt)) { task.status = str.replaceAll(/.*\(status=(.*)\).*/, { all, status -> return status }) } }}
Olaf David, CSU/USDA 43
Examples*Problem elevationSCRUM supportSCM commit controlAuditable Build ManagementTracker Status Statistics
* The examples demonstrate the support of some aspects of the topics above
Olaf David, CSU/USDA 44
Objective◦ Flexible build process triggering by tracker content
information◦ Capture all information that relate to a build
Bugs <-> Build Features <-> Build Build process log <-> Build Build product <-> Build
Benefit◦ Developer/PM independent build control◦ Build result notification◦ Automated build product storage and versioning◦ Auditable and traceable
Auditable Build Management
Olaf David, CSU/USDA 45
A project
Bugs
FeatureRequests
DEV
SCMRepo
CommitUpdatePushPull
Pick upEditCommentStatus change
Commit assoc.
Fixed
Implemented
Developer changes task status to..
The developer’s view
Olaf David, CSU/USDA 46
Login and get the project reference
Auditable Build Management (1)
import cbscript.*import static cbscript.CB.*
// login to CBjf = CB.login("http://localhost:8080", "bond", “pass")
// find the project of interestprj = jf.projects.find{it.name == "OMSProject"}
...
Olaf David, CSU/USDA 47
A project Local
Bugs
FeatureRequests
Milestones
DEV
SCM
Build-lib
Script
(2)
Documents
MGR
Identify
Olaf David, CSU/USDA 48
Find ‘candidates’ in ‘Bugs’ and ‘Features’ that should lead to a new build
Combine the lists in to one.
Auditable Build Management (2)...
// normal and higher priority bug fixed this monthreadyBugs = prj.trackers.find {it.name == "Bug"}.issues.findAll { it.status == "Fixed" && isThisMonth(it.submitted) && it.priority > LOW }
// A feature is implemented and there is a commitreadyFeatures = prj.trackers.find { it.name == “Features"}.issues.findAll { it.status == "Implemented" && it.hasCommits}
// combine all features and bugs into one single listreadyList = readyBugs + readyFeatures...
Olaf David, CSU/USDA 49
A project Local
Bugs
FeatureRequests
Milestones
DEV
SCM
Build-lib
Script
Documents
3.1 checkout/update
(3)
3.4 <cbrelease>
3.2 build
3.3 log
MGR
Issue build
Olaf David, CSU/USDA 50
If the list is not empty, we need to build Find the build and invoke it. The build will create the file ‘builds/ngmf.jar’
in ‘Documents’
Auditable Build Management (3)...
// build checkif (!readyList.empty) {
// invoke the build. build = prj.builds.find{it.name == "build-lib"} result = build.invoke();
...
Olaf David, CSU/USDA 51
A project Local
Bugs
FeatureRequests
Milestones
DEV
SCM
Build-lib
Script
Documents
(4) MGRNew milestone build
Olaf David, CSU/USDA 52
Create a WIKI description with the result info (status and link to the Buildlog)
Submit a new milestone task with the WIKI content.
Notification will be sent on submission.
Auditable Build Management (4)...
String wikiinfo = "!!Milestone info \n" + "!Milestone Build:\n" + "* Build Status: ${result.status}\n" + "* Output Log Reference: ${CB.ref(result)}\n"
// create a new milestone in the milestone tracker mt = prj.trackers.find {it.name == "Milestones"} milestone = mt.submit( new Task(name:"MS-1.2", text:wikiinfo, priority:LOW))...
Olaf David, CSU/USDA 53
A project Local
Bugs
FeatureRequests
Milestones
DEV
SCM
Build-lib
Script
Documents
(5)
5.1 relate / link
5.2 relate
MGR
Olaf David, CSU/USDA 54
Relate the new milestone to all bugs and features in the readyList.
Also relate it to the build product (ngmf.jar)
Auditable Build Management (5)
... // the build product def lib = prj.artifact("builds/ngmf.jar")
// relate the milestone to all // bugs/features/and the build product milestone.relate(lib, RELATED) readyList.each{ task -> milestone.relate(task, DEPENDS) }...
Olaf David, CSU/USDA 55
A project Local
Bugs
FeatureRequests
Milestones
DEV
SCM
Build-lib
Script
Documents
(6)
6.2 copy
MGR
Fixed -> ReadyForTesting
Implemented -> ReadyForTesting
6.1 Task Status change ..
ngmf.jar
Olaf David, CSU/USDA 56
Copy the build product to the local FS for testing to skip manual download.
Change the status for all bugs and features to “ReadyForTesting” to indicate their processing
Auditable Build Management (6)
... // copy the new lib from CB to my local FS (optional) new File('c:/tmp/ngmf.jar').withOutputStream { it.write lib.bytes }
// change the status in all issues in readylist to // "ReadyforTesting" readyList.each{ task -> task.status = "ReadyForTesting" }}
Olaf David, CSU/USDA 57
Before invocation and …
Olaf David, CSU/USDA 58
… After …
Reference to Build
Status change
Olaf David, CSU/USDA 59
.. and After ..
Reference to Bugs and Features
Reference to build product
Status
Link to BuildLog
Olaf David, CSU/USDA 60
.. and After ..
Reference to Milestone
build
Build Product
Olaf David, CSU/USDA 61
.. and After ..
New Milestone
Build
Milestones
All associated
Bugs and features
Olaf David, CSU/USDA 62
Build process locally executed Upload the build product as a milestone
attachment to the server Upload the local build console output to the
milestone as comment … rest stays the same.
Variation (Local Build)
Olaf David, CSU/USDA 63
Execute the build on the local machine Capture the process output
Variation (cont.)...
// build checkif (!readyList.empty) {
sout = new StringBuffer() serr = new StringBuffer()
// invoke the build. p = “make –f /home/me/makefile all”.execute() p.consumeProcessOutput(sout, serr) result = p.waitFor()
...
Olaf David, CSU/USDA 64
Attach the build product to the milestone and the console output as comment
Variation (cont.)... String wikiinfo = "!!Milestone info \n" + "!Milestone Build:\n" + "* Build Status: ${result}\n"
// create a new milestone in the milestone tracker mt = prj.trackers.find {it.name == "Milestones"} milestone = mt.submit( subject:"MS-1.2", text:wikiinfo, priority:LOW) if (result==0) milestone.attach(“/tmp/libngmf.so”) milestone.comment(text: result == 0 ? sout : serr)...
Olaf David, CSU/USDA 65
Examples*Problem elevationSCRUM supportSCM commit controlAuditable Build ManagementTracker Status Statistics
* The examples demonstrate the support of some aspects of the topics above
Olaf David, CSU/USDA 66
Objective◦ Get the status distribution of all issues in all
tracker of a given project◦ Visualize results ad-hoc
Benefit◦ Get a quick overview about the completeness and
work status across tracker
Tracker Status Distribution
Olaf David, CSU/USDA 67
Create a map with each key as status choice and the value for the # of issues with this status.
Tracker Status Distribution (1)def bugs = prj.trackers.find{it.name == "Bug"} // get the bug tr.
def smap = [:] bugs.statusChoices.each{ smap[it]=0 } // (key:status, value:0)
bugs.issues.each { bug -> smap[bug.status]++ } // fill the map
println smap // print the content
// ["New":20, "Unconfirmed":10, "Verified":5, "Resolved":5, "Reopened":0, "Closed":20, "ReadyForTesting":3]
Olaf David, CSU/USDA 68
Graph the result using JFreeChart and SwingBuilder
Tracker Status Distribution (2)// data setdef data = new DefaultPieDataset();smap.each{k, v -> data.setValue "$k", v } // smap to dataset
// chartdef chart = ChartFactory.createPieChart(“Bug Tracker Status", data,
false, true, true)chart.plot.labelGenerator = new LG(' {0} - {1} ({2}) ')// framedef frame = new SwingBuilder().frame(title:’PieChart',
defaultCloseOperation:WC.EXIT_ON_CLOSE) { widget(new ChartPanel(chart))}frame.pack()frame.show()
Olaf David, CSU/USDA 69
Tracker Status Distribution (3)
Olaf David, CSU/USDA 70
http://cbscript.javaforge.com◦ CB script examples, download cbscript.jar
https://codebeamer.com/cb/wiki/18830◦ Codebeamer API
http://groovy.codehaus.org◦ All about Groovy
https://codebeamer.com◦ Codebeamer
References
top related