business process automation with codebeamer & groovy (updated)

Groovy Business Process

AutomationO. David

Colorado State UniversityDept. of Civil Engineering, Dept. of Computer Science

US Department of Agriculture

Olaf David, CSU/USDA 2

Automate and integrate software project management practices using CB with respect to◦ Trackers◦ Forums◦ Builds◦ Documents◦ Users

by using an efficient scripting approach.

Objective


A First Exampleimport cbscript.CBimport static cbscript.CB.*import org.codehaus.groovy.runtime.TimeCategory

jf = CB.login("http://localhost:8080", “bond", “pass")proj = jf.projects.find{it.name == "MyProj"}

use(TimeCategory) { needsBuild = proj.trackers.find{it.name ==“Bugs"}.issues.any{ bug -> bug.status == "Fixed" && bug.submitted < 30.days.ago && bug.priority > LOW && bug.hasCommits }

if (needsBuild) { println “Starting build ....“ log = proj.builds.find{it.name == “BuildApp"}.invoke() println " Build done: ${log.status} “ println " Output: ${log.stdOut} " }}

jf.logout();


Principal Architecture

cbscript

cb-api CB Server

DB SCM

Java API- Low level, session centric, stateless- For a wide range of applications

Groovy API- High level, object centric, and productivity oriented- For process automation

Docs


Semantic density and flexibility of scripts◦ Usability/Efficiency aspects of scripts vs. plain Java

vs. Web UI Remote Management of (multiple) CB server Leveraging general scripting productivity

(Groovy)◦ Closures, Properties, Duck typing◦ Builder (DSL)◦ Intuitive Date management/use

CB web UI does not allow complex queries No automation using the CB Web UI

Why Scripting?


Java 1.6+ Groovy 1.6+ cbscript-5.3.1.jar

◦ Contains cb-api.jar of the same CB version CB 5.3.x

◦ Network access to context ‘<cburl>/cb/remote-api’

◦ Project user access (account)

Prerequisites


API respects CB licenses, access permissions

Login/Logoutimport cbscript.CBimport static cbscript.CB.*

jf = CB.login("http://localhost:8080", “bond", "007")// orjf = CB.login( url: "http://10.177.19.63:8080", login: "bond", password:“pass")

// do some work.

jf.logout();


meta information can be obtained about the session.

read/only

Session informationassert jf.login == "bond"assert jf.url == "http://localhost:8080"assert jf.user.realName == "Default System Administrator"


‘projects’ returns a list of projects Using closures to iterate, filter, search,…

Getting Projects// list all projectsjf.projects.each { println " id: ${it.id} name : ${it.name}"}

// find project(s)assert jf.projects.find{it.name == "Moose"}.id == 2374assert jf.projects.find{it.id == 2374}.name == "Moose“

// directjf.project(‘Moose’)


‘trackers’ returns a list of tracker Use closures to iterate, filter, search

Managing Tracker// get all tracker of the ‘oms’ project.jf.projects.find{it.name == "oms"}.trackers.each { tracker -> print tracker.name print tracker.description }


Closures help traversing resources Obtain the Bug tracker in the oms project.

Then find all the ‘Closed’ issues

Tracker Issues query// Find issues that are not Closeddef bugTracker = jf.projects.find{ it.name == "oms“ }.trackers.find {it.name == "Bugs"}

bugTracker.issues.findAll{ it.status != 'Closed'}.each{ println it.name}


Find all issues assigned to me and submitted last week

Tracker Issues query (cont.)bugTracker.issues.findAll{ it.assignedTo == jf.user && it.status == 'New'}.each { println it.name}


Find a tracker issue that has the string 'big problem' in summary

Tracker Issues query (cont.)bugTracker.issues.findAll{ it.summary.contains('big problem')}.each{ println it.name}


All 'Resolved' issues assigned to user 'bond' that were submitted the last 10 days

Groovy date use is intuitive!

Tracker Issues query (cont.)...bugTracker.issues.findAll{ it.status == "Resolved" && it.assignedTo.name == "bond" && it.submitted < 10.days.ago}.each{ println it.name}


All tracker issues submitted by me and do not have an attached SCM commit.

Tracker Issues query (cont.)bugTracker.issues.findAll{ return it.submitter == jf.user && !it.hasCommits}.each{ println it.name}


All tracker issues with high priority that are still open and are not assigned yet to a developer.

Tracker Issues query (cont.)bugTracker.issues.findAll{ bug -> bug.priority == HIGH && bug.status != “Closed” && bug.assigned == null}.each{ println it.name}


Use properties ‘summary’, ‘text’, ‘priority’… Priority: LOWEST, LOW, NORMAL, HIGH,

HIGHEST

Submit a new Issue// get the bug trackerdef bugs = jf.projects.find{ it.name == “oms"}.trackers.find {it.name == "Bugs"}

bugs.submit(summary:"My Bug", text:"This is a new Bug", priority:NORMAL)


Comment on a existing issue.

Comment on an Issue

bug = bugs.submit(summary:"My Bug", text:"This is a new Bug", priority:NORMAL)

bug.comment(text:”please specify”, format:PLAIN_TEXT)


Upload and attach a file to an issue. (file is not stored in Documents)

Attach a file to an Issue

bug = bugTracker.issues.find{ bug -> bug.priority == HIGH}

bug.attach(file:”/tmp/spec-1.txt”)


‘forums’ contains all project forums Check for new posts today.

Forums

def newsForum= jf.projects.find{ it.name == "oms"}.forums.find {it.name == "News"}

newsForum.findAll{ post -> post.submitted > 8.hours.ago}.each{ print it.text}


‘forums’ contains all project forums Check for new posts today.

Post a new Forum message

news= jf.projects.find{it.name == "oms"}.forums.find { it.name == "News“}

news.post(subject:"New message", text:"Content here ...")


Check for all new posts from “Joe Poster” submitted this week and print out the message text.

Check for Posts

news.findAll{ post -> post.submitter.realName == “Joe Poster” && post.submitted < 1.week.ago}.each{ print it.text}


Use ‘artifact’ to reference a document or folder in the project ‘Documents’

Always absolute to ‘Documents’ root.

Documents

prj = jf.projects.find{it.name == "cbtest"}doc = prj.artifact(‘download/EC2TW.txt’)

assert doc.bytes.size() == 709assert doc.name == ‘EC2TW.txt’assert doc.directory == false


Copy a file from a CB server to the local FS

Documents (cont.)

prj = jf.projects.find{it.name == "cbtest"}doc = prj.artifact("download/EC2TW.txt")

new File('c:/tmp/test1').withOutputStream { it.write doc.bytes}


‘builds’ contains all the builds of a project. Call ‘invoke’ to execute the build on a CB

server. (Invoke is a synchronous call)

Builds build = prj.builds.find{it.name == "build-lib"} log = build.invoke();


Build ‘log’ captures the result of the build.

Builds (cont.)build = prj.builds.find{it.name == "build-lib"}log = build.invoke();

assert log.output != nullassert log.error == nullassert log.status == “Successful”assert log.successful == trueassert log.startedBy == jf.user


Create associations by calling ‘relate’ on tasks, artifacts, documents, etc.

Pass in the ‘other’ task, artifact, document and the kind of relationship (RELATED, CHILD, PARENT, DEPENDS)

Associations

bug = bugTracker.submit(summary:"My Bug", text:"This is a new Bug", priority:HIGH)

def lib = prj.artifact("builds/ngmf.jar")

bug.relate(lib, RELATED)


Create a Association to many issues in one ‘bulk’ operation (use closure)

‘milestone’ is a tracker issue

Associations (cont.)

readyToBuild = prj.trackers.find { it.name == "Bug“}.issues.findAll { it.status == "Fixed" && thisMonth(it.submitted) }

readyToBuild.each { bug -> milestone.relate(bug, DEPENDS) }


Use ‘CB.link()’ to create interwiki links for task messages, forum posts, artifact descriptions to reference other resources

InterWiki linksbuild = prj.builds.find{it.name == "build-lib"}result = build.invoke();

String wikitext = “””!!Milestone info* Output Log Reference: ${CB.link(result)}“””

// resolves to:// “ Output Log Reference : [BUILDLOG:23456]”


Direct access to a project/forum/tracker/issue/post/build/.. using its ‘Id’ or ‘name’ (projects only)

No traversing all higher level resources necessary -> very efficient, but you need to know the ID!!!

Resource Access Efficiency vs. Query Flexibility

Efficient and Direct Accessprj = jf.project(‘moose’) // get a project by nameprj = jf.project(1234) // get a project by If

tra = jf.tracker(2345) // get a tracker by idiss = jf.issue(23467) // get a issue by idbld = jf.build(23411) // get a build by id...


Examples*Problem elevationSCRUM supportSCM commit controlAuditable Build ManagementTracker Status Statistics

* The examples demonstrate the support of some aspects of the topics above


Objective◦ Increase the priority of issues, based on some

dynamic rules

Benefit◦ Early problem detection, identification, and

communication

SCRUM support


Increase the priority of all verified bugs submitted before April that are assigned to Joe that he did not fixed… so far.

Call this above every day (e.g. cron)

Simple Problem Elevationbugs.findAll{ bug -> bug.status == “Verified" && bug.assignedTo.name == “Joe” && bug.spendHours == 0 && !bug.hasCommits && bug.submittedAt < new Date(2009,4,1)}.each{ it.priority++}


Objective◦ Support some aspects of SCRUM

Benefit◦ SCRUM coverage◦ Sprint meeting preparation

SCRUM support


Prepare a Sprint review meeting◦ Check which tasks in 'Feature' requests in the

'oms' project were finished over the last 30 days.

SCRUM Support

jf.projects.find{it.name == "oms"}.trackers.find{ it.name == "Features"}.issues.findAll { task -> task.status == "Closed" && task.modifiedAt > 30.days.ago}.each { println "${it.summary} - ${it.description}"}


Prepare a Sprint review meeting◦ Get all the bugs that are still not closed and were

submitted since April 1st 2008 (last sprint meeting)

SCRUM support (cont.)

jf.projects.find{it.name == "oms"}.trackers.find {it.name == "Bugs"}.issues.findAll{ it.status != "Closed" && it.submittedAt > new Date(2008, 1, 4)}.each { println "${it.summary} - ${it.description}"}


Creating a sprint backlog for all closed tasks in the UI category of new features.◦ The map contains the last seven days,

key: day diff to today (-1 is yesterday) value: the number of closed tasks

SCRUM Support (cont.)def issues = jf.projects.find{it.name=="oms"}.trackers.find{ it.name == "Features"}.issues.findAll { it.category == ‘UI’}(-7..0).each { day -> map[day] = issues.findAll { it.status == "Closed" && it.modifiedAt < 8.hours.ago }.size()}

println map// [-7:10, -6:3, -5:12, -4:4, -3:10, -2:5, -1:1, 0:2]


Creating daily burndown chart data ◦ Call this script every day to track the hours

being ‘burned’ in developing new features .◦ Create the graph based on this file.

SCRUM Support (cont.)hours = 0def fissues = jf.projects.find{it.name=="oms"}.trackers.find{ it.name == "Features“}.issues.findAll { it.status == “InDevelopment”}.each { hours += it.estimatedHours – it.spentHours}

f = new File(“burndown.dat”)today = new Date()f.append("${today} , ${hours}")


Objective◦ A commit contains additional (meta) information

that triggers process actions such as an issue status change

Benefit◦ This developer’s shortcut saves time

General Pattern for triggering of process actions by commit message content

SCM Commit triggers status change


A SVN commit message such as

svn commit –m “#1234 (status=Fixed) Changed loop order” will trigger a status change on a task to fixed; no separate UI

operation needed. (Put this in cron and execute frequently)

SCM Commit triggers status change

jf.projects.find{it.name == "OMSProject"}.trackers.find { it.name == "Bugs“}.issues.findAll{ it.hasCommits && it.status != "Closed“}.each { task -> task.commits.each { if (it.message ==~ /.*$status=(.*)$.*/ && today(it.commitedAt)) { task.status = str.replaceAll(/.*$status=(.*)$.*/, { all, status -> return status }) } }}


Objective◦ Flexible build process triggering by tracker content

information◦ Capture all information that relate to a build

Bugs <-> Build Features <-> Build Build process log <-> Build Build product <-> Build

Benefit◦ Developer/PM independent build control◦ Build result notification◦ Automated build product storage and versioning◦ Auditable and traceable

Auditable Build Management


A project

Bugs

FeatureRequests

DEV

SCMRepo

CommitUpdatePushPull

Pick upEditCommentStatus change

Commit assoc.

Fixed

Implemented

Developer changes task status to..

The developer’s view


Login and get the project reference

Auditable Build Management (1)

import cbscript.*import static cbscript.CB.*

// login to CBjf = CB.login("http://localhost:8080", "bond", “pass")

// find the project of interestprj = jf.projects.find{it.name == "OMSProject"}

...


A project Local

Bugs

FeatureRequests

Milestones

DEV

SCM

Build-lib

Script

(2)

Documents

MGR

Identify


Find ‘candidates’ in ‘Bugs’ and ‘Features’ that should lead to a new build

Combine the lists in to one.

Auditable Build Management (2)...

// normal and higher priority bug fixed this monthreadyBugs = prj.trackers.find {it.name == "Bug"}.issues.findAll { it.status == "Fixed" && isThisMonth(it.submitted) && it.priority > LOW }

// A feature is implemented and there is a commitreadyFeatures = prj.trackers.find { it.name == “Features"}.issues.findAll { it.status == "Implemented" && it.hasCommits}

// combine all features and bugs into one single listreadyList = readyBugs + readyFeatures...


A project Local

Bugs

FeatureRequests

Milestones

DEV

SCM

Build-lib

Script

Documents

3.1 checkout/update

(3)

3.4 <cbrelease>

3.2 build

3.3 log

MGR

Issue build


If the list is not empty, we need to build Find the build and invoke it. The build will create the file ‘builds/ngmf.jar’

in ‘Documents’


// build checkif (!readyList.empty) {

// invoke the build. build = prj.builds.find{it.name == "build-lib"} result = build.invoke();

...


A project Local

Bugs

FeatureRequests

Milestones

DEV

SCM

Build-lib

Script

Documents

(4) MGRNew milestone build


Create a WIKI description with the result info (status and link to the Buildlog)

Submit a new milestone task with the WIKI content.

Notification will be sent on submission.


String wikiinfo = "!!Milestone info \n" + "!Milestone Build:\n" + "* Build Status: ${result.status}\n" + "* Output Log Reference: ${CB.ref(result)}\n"

// create a new milestone in the milestone tracker mt = prj.trackers.find {it.name == "Milestones"} milestone = mt.submit( new Task(name:"MS-1.2", text:wikiinfo, priority:LOW))...


A project Local

Bugs

FeatureRequests

Milestones

DEV

SCM

Build-lib

Script

Documents

(5)

5.1 relate / link

5.2 relate

MGR


Relate the new milestone to all bugs and features in the readyList.

Also relate it to the build product (ngmf.jar)


... // the build product def lib = prj.artifact("builds/ngmf.jar")

// relate the milestone to all // bugs/features/and the build product milestone.relate(lib, RELATED) readyList.each{ task -> milestone.relate(task, DEPENDS) }...


A project Local

Bugs

FeatureRequests

Milestones

DEV

SCM

Build-lib

Script

Documents

(6)

6.2 copy

MGR

Fixed -> ReadyForTesting

Implemented -> ReadyForTesting

6.1 Task Status change ..

ngmf.jar


Copy the build product to the local FS for testing to skip manual download.

Change the status for all bugs and features to “ReadyForTesting” to indicate their processing


... // copy the new lib from CB to my local FS (optional) new File('c:/tmp/ngmf.jar').withOutputStream { it.write lib.bytes }

// change the status in all issues in readylist to // "ReadyforTesting" readyList.each{ task -> task.status = "ReadyForTesting" }}


Before invocation and …


… After …

Reference to Build

Status change


.. and After ..

Reference to Bugs and Features

Reference to build product

Status

Link to BuildLog


.. and After ..

Reference to Milestone

build

Build Product


.. and After ..

New Milestone

Build

Milestones

All associated

Bugs and features


Build process locally executed Upload the build product as a milestone

attachment to the server Upload the local build console output to the

milestone as comment … rest stays the same.

Variation (Local Build)


Execute the build on the local machine Capture the process output

Variation (cont.)...

// build checkif (!readyList.empty) {

sout = new StringBuffer() serr = new StringBuffer()

// invoke the build. p = “make –f /home/me/makefile all”.execute() p.consumeProcessOutput(sout, serr) result = p.waitFor()

...


Attach the build product to the milestone and the console output as comment

Variation (cont.)... String wikiinfo = "!!Milestone info \n" + "!Milestone Build:\n" + "* Build Status: ${result}\n"

// create a new milestone in the milestone tracker mt = prj.trackers.find {it.name == "Milestones"} milestone = mt.submit( subject:"MS-1.2", text:wikiinfo, priority:LOW) if (result==0) milestone.attach(“/tmp/libngmf.so”) milestone.comment(text: result == 0 ? sout : serr)...


Objective◦ Get the status distribution of all issues in all

tracker of a given project◦ Visualize results ad-hoc

Benefit◦ Get a quick overview about the completeness and

work status across tracker

Tracker Status Distribution


Create a map with each key as status choice and the value for the # of issues with this status.

Tracker Status Distribution (1)def bugs = prj.trackers.find{it.name == "Bug"} // get the bug tr.

def smap = [:] bugs.statusChoices.each{ smap[it]=0 } // (key:status, value:0)

bugs.issues.each { bug -> smap[bug.status]++ } // fill the map

println smap // print the content

// ["New":20, "Unconfirmed":10, "Verified":5, "Resolved":5, "Reopened":0, "Closed":20, "ReadyForTesting":3]


Graph the result using JFreeChart and SwingBuilder

Tracker Status Distribution (2)// data setdef data = new DefaultPieDataset();smap.each{k, v -> data.setValue "$k", v } // smap to dataset

// chartdef chart = ChartFactory.createPieChart(“Bug Tracker Status", data,

false, true, true)chart.plot.labelGenerator = new LG(' {0} - {1} ({2}) ')// framedef frame = new SwingBuilder().frame(title:’PieChart',

defaultCloseOperation:WC.EXIT_ON_CLOSE) { widget(new ChartPanel(chart))}frame.pack()frame.show()


Tracker Status Distribution (3)


http://cbscript.javaforge.com◦ CB script examples, download cbscript.jar

https://codebeamer.com/cb/wiki/18830◦ Codebeamer API

http://groovy.codehaus.org◦ All about Groovy

https://codebeamer.com◦ Codebeamer

References

http://cbscript.javaforge.com/

https://codebeamer.com/cb/wiki/18830

http://groovy.codehaus.org/

https://codebeamer.com/

business process automation with codebeamer & groovy (updated)

Documents

bug bug

bug tracker

projects jf

closed issues olaf david

summary olaf david

logout olaf david

documents olaf david

search olaf david