combining rule-based and information retrieval techniques to assign software change requests...

Combining Rule-based and InformationRetrieval Techniques to assign Software

Change RequestsNew Idea

Yguarata Cerqueira CavalcantiIvan do Carmo Machado

Paulo A. da Mota S. NetoEduardo Santana de Almeida

Silvio Romero de Lemos Meira

Center for Informatics – UFPE, Brazil

ASE 2014 (Vasteras – Sweden)September 18, 2014

Agenda

1 Introduction

2 Proposal

3 Experiment

4 Concluding remarks

1 Introduction

2 Proposal

3 Experiment


Outline Introduction Proposal Experiment Concluding remarks References

About Change Requests (CRs)

Change Request (CR) describes a defect to be fixed, an adaptivechange, or a new functionality.

CRs are stored and managed through CR Repositories.

1/14 Yguarata Cerqueira Cavalcanti Combining Rule-based and Information Retrieval Techniques to assign Software Change Requests


CR Assignment

CR Assignment consists of matching available developers withreported CRs, considering low fixing time and highquality.

Why CR Assignment Matters?

Dozens to hundreds CRs daily [1]

Needs good knowledge on the project [2]

Despite the number of semi-automated approaches, it is stillperformed manually

Labor-intensive, time consuming, susceptible to mistakes [2]



Motivation

Current approaches use AI. . .

Performs as a black box

Difficult adoption

Hard to deal with changing environments


1 Introduction

2 Proposal

3 Experiment



New Idea

Semi-automation through

Combining Rule-based andInformation Retrieval Techniques

Aided by context information



Strategy to Automated CR Assignment

New CR is submitted

First step:

Rule-Based ExpertSystem (RBES) runs oversimple rules (low cost)

Second step

Information Retrieval (IR)model runs and extractspotential developersRBES runs over complexrules (high cost)



Rule-Based Expert System (RBES) – Drools

Simple rule

rule "Critical CRs, or CRs for module C"

when

$cr: CR(severity == CRITICAL || module =="C")

then

$cr.assignTo(developer("[email protected]"))

end

Complex rule

rule "Change Requests for modules A and B"

when

$cr: CR(module =="A" || module =="B")

then

$cr.assignTo(availableDeveloper(Workload.WEIGHTED ))

end



Benefits

Fully controlled

Developers can intervene

Easy adoption

Context information promotes company dynamics


1 Introduction

2 Proposal

3 Experiment



Questions

Q1: What is the accuracy of the proposed approach forautomated CR assignment?

Q2: What is the necessary effort to setup the approach ina software development project?

Q3: Does the achieved accuracy pay the necessary effortneeded in the setup?



Experiment Design

Proposed approach versus pure Support VectorMachine (SVM)

CRs from two modules of NovoSIAFIBrazilian Federal Data Processing Service (SERPRO)

Module A = 781 CRs

Module B = 1031 CRs



Configuration of the Proposed Approach

Rules extraction

Total of 14 rules

Context information

developers vacation

developers project allocation

developers experience



Results I

Q1. What is the accuracy of the proposed approach forautomated CR assignment?

Our approach: Module A = 45% and Module B = 34%

SVM: Module A = 38% and Module B = 23%

An improvement of 18% on Module A and 48% on B



Results II

Q2. What is the necessary effort to setup the approach in asoftware development project?

38 hours (rule extraction, context information, strategy)

Q3. Does the achieved accuracy pay the necessary effortneeded in the setup?

Considering 10 minutes for each CR manually assigned [2]

SVM would save 89 hours

Our approach would save 117 hours

Thus, an economy of 28 hours

However, 38 hours was needed for setup


1 Introduction

2 Proposal

3 Experiment



Concluding remarks

RBES + IR: simple, yet effective !

Future work

Investigate new algorithms for workload balancing

Investigate methods and techniques for automatic extraction ofassignment rules


Combining Rule-based and InformationRetrieval Techniques to assign Software

Change RequestsNew Idea

Yguarata Cerqueira CavalcantiIvan do Carmo Machado

Paulo A. da Mota S. NetoEduardo Santana de Almeida

Silvio Romero de Lemos Meira

Center for Informatics – UFPE, Brazil

ASE 2014 (Vasteras – Sweden)September 18, 2014


References I

[1] Cavalcanti, Y. C., da Mota Silveira Neto, P. A., Lucredio, D.,Vale, T., de Almeida, E. S., and de Lemos Meira, S. R. (2013a).The bug report duplication problem: an exploratory study. Soft.Quality J., 21, 36–66.

[2] Cavalcanti, Y. C., Neto, P. A. M. S., Machado, I. C., Almeida,E. S., and Meira, S. R. L. (2013b). Towards UnderstandingSoftware Change Request Assignment: A survey withpractitioners. In Proc. of the 17th Inter. Conf. on Evaluation andAssessment in Soft. Eng. (EASE).

[3] Jeong, G., Kim, S., and Zimmermann, T. (2009). Improving bugtriage with bug tossing graphs. In Proc. of the 7th joint meetingof the European Soft. Eng. Conf. and the ACM SIGSOFT Symp.on the Found. of Soft. Eng. (ESEC/FSE’2009), pages 111–120.


CR Assignment


Information Retrieval Model With Learning

SVM

Training (Black arrows)

Recommendation (Gray arrows)


Component Diagram of the Solution


Prototype Tool Architecture
