which feature location technique is better?
Post on 04-Feb-2016
26 Views
Preview:
DESCRIPTION
TRANSCRIPT
Which Feature Location Technique is Better?
Emily Hill, Alberto Bacchelli, Dave Binkley, Bogdan Dit,
Dawn Lawrie, Rocco Oliveto
Motivation: Differentiating FLTs
Totally unrelated
In vicinity
Precision = 0.20 Precision = 0.20
Example• Developer works down ranked list• At each item can explore or not• When exploring structure, can bail
at any time
Proposed Approach: Rank Topology
• Use evaluation measures that consider the likelihood of a developer finding fix locations
• Use textual information to approximate developer’s interest (i.e., likelihood) of following “trail” in structural topology, starting from ranked list
• Rank topology = inverse of the number of hops in topology
Example• Developer works down ranked list• At each item can explore or not
• 3rd rank result + 4 structural hops = 7 total hops
• Rank topology metric = 1 / 7
• No discrimination: explores everything
How “smart” is the user?
• Semi-intelligent: only follows a structural hop if the next method exhibits textual clues– Rank topology uses VSM cosine similarity (tf-idf)– Structural edge added if both methods > median
scores for query– Supported by user studies of information foraging
theory [Lawrance, et al TSE 2013]
• Omniscient: makes no wrong choices, exploring only those ranks and structural hops that lead to a bug
Preliminary Study: Distinguish QLM from Random
Ranked list of results all have same bug fixes at exactly the same ranks
Conclusion• Rank topology differentiates between
randomly ordered lists and a state of the art IR technique (QLM) with relevant results at the exact same ranks
• Future work– How well does rank topology mimic developer
behavior in practice?– How closely can/should we model user behavior?
• Our question: Does the research community need to revise how we evaluate FLTs?
Preliminary Study
• Effect of program structure on the rank topology metric for each JabRef bug used in the case study.
top related