a functional software measurement approach bridging the gap between problem and solution domains ...
TRANSCRIPT
A FUNCTIONAL SOFTWARE MEASUREMENT APPROACH BRIDGING THE GAP BETWEEN PROBLEM AND SOLUTION DOMAINS
by Dr.Erdir Ungan, Prof.Dr. Onur Demirörs
Problem
• Granularity • Issues in Parametric Es3ma3on Methods • Issues in Benchmarking • Reliability Issues in FSM Measurements • Issues Regarding Measurement Procedure
Anything you try to quan.fy can be divided into any number of "anythings," or become the thing -‐ the unit -‐ itself. And what is any number, itself, but just another unit of measurement?
What is a 'six' but two 'threes', or three 'twos'...half a 'twelve', or just six 'ones' -‐ which are what?
-‐ F.L. Vanderson
Issues in Parametric EsJmaJon Methods
• Most es3ma3on methods suggest ◦ predic3ng soEware size ◦ Measure PD size and convert it to SD size based on some historical data. ◦ ….Huge varience.
• The process of developing a solu3on to a problem is ambiguous and soE. Therefore, es3ma3on methods in PD need subjec3ve input ◦ This is one of the points where the reliability and repeatability of these methods are ques3oned.
Issues in Benchmarking
• Problems in measurement results and granularity also affects the quality of data in benchmarking data sets.
• Özcan Top and Yilmaz concluded that, those sets lack structural informa3on about the projects
• We cannot deduct informa3on about the abstrac3on level of measurements in those data sets
• Comparing data from varying abstrac3on levels will result in erroneous benchmarking
SeparaJon of Problem and SoluJon Domains
The difference between theory and prac2ce equals your inep2tude.. .
Anonymous
What vs. How
◦ How, is the process of breaking down a larger What to smaller What’s.
◦ What1 ◦ What 1.1 ◦ What 1.2
◦ What 1.2.1 ◦ What 1.2.2
◦ What 1.2.3 ◦ What 1.2.3.1 ◦ . ◦ .
◦ One of these transi3ons in the W-‐H chain will pose as a boundary between Problem and Solu3on domains based on the defini3on of the system at hand.
What vs. How • Increase Profit • … • Decrease Costs • Increase Sales ◦ ... ◦ Increase Adver3sement ◦ Establish an online store
◦ … ◦ Get a Server ◦ Build an e-‐commerce site
◦ Design Web Site ◦ Test Web site ◦ Develop Web Site
◦ Develop Interface ◦ Develop Client Side SoEware ◦ Develop Server Side SoEware
• Develop Database • Develop Data Abstrac3on Layer • Develop Business Layer
◦ Develop Admin Module ◦ Develop Stock Module ◦ Develop Customer Module
◦ Develop membership func3ons ◦ Develop CRM func3ons
◦ Develop Customer Class ◦ Develop Purchases Class
◦ Develop add purchase method ◦ Develop get purchasename method
PD & SD in SoQware Engineering
Mutual Independence of PD & SD
• Although common sense and experience dictates that PD and SD are two separate domains. We needed to demonstrate this phenomenon.
• We conducted two case studies ◦ First Study: We inves3gated the correla3on of problem and solu3on domain sizes with the problem and solu3on domain effort
◦ Second Study: We conducted a case study where a single set of requirements were to be developed using two different implementa3on approaches
• Both studies supported our sugges3ons about problem and solu3on domain separa3on
Mutual Independence of PD & SD
Bytes / CFP Std. Dev. R2 Comfort & Convenience Type Components 7.6 0.99
Display & Indica3on Type Components 21.6 0.992
For single organiza3on’s data set, varia3on and correla3on in Physical size (Bytes) to
COSMIC size is given in
(Gencel, Heldal, and Lind 2009)
SLOC / CFP Std. Dev. R2 Comfort & Convenience Type Components 8.2 0.4855
15.4 0.417
For single organiza3on’s data set, varia3on and correla3on in SLOC sizes to COSMIC
Context
Context
◦ We define Gaps 1-‐2-‐3-‐5 and 6 as imperfec3ons and therefore evitable.
◦ However, we believe, ideally there is: ◦ One correct and exact defini3on of the real life problem. ◦ One correct set of requirements that describe the problem. Conforming to a requirements standard.
◦ One correct set of specifica3ons that breaks down the requirements to lower level specifica3ons.
◦ One best implementa3on of the design based on the technology and implementa3on method used.
◦ One best and unique soEware build for the implementa3on. Through an ideal compiler.
Context
• However, Gap 4, the gap between specifica3ons and design is different in nature from these gaps. “There is no one correct solu3on to a problem.”
• Engineering solu3ons are guided by many principles, design palerns and rules. Nevertheless, there is always a subjec3ve, crea3ve element in crea3ng solu3ons to problems. Therefore this gap is essen3ally inevitable.
• The transforma3on between specifica3ons and design is actually where we define the border between the problem domain and solu3on domain is.
Context
SoluJon Approach
SoluJon Approach
• U3lize a Common Concept For PD & SD • Es3ma3on Accuracy is Based on Level of Detail • Incorpora3ng Decomposi3on Level (Granularity) Into Measurement • Defining an Absolute Reference Point
• Automated Measurement • Minimizing Interpreta3ons and Assump3ons
Proposed Method
• We propose a two dimensional measurement: ◦ Data movement ◦ Decomposi3on Level
Therefore, we define each aspect of a measurement method in these two dimensions.
Conceptual Modeling
Increasing Decomposi3on Decreasing Tier #
Tier • Each decomposi3on of the system will result in a new Tier of system defini3on. Each increase in the 3er number represents one higher level of decomposi3on traversed in the descrip3on of the system.
• Each 3er consists of structural en33es communica3ng with same 3er level of en33es.
Note: One set of objects in a 3er can be present in a higher level if the entry and exit point of data movements between them and other objects does not change on their end. That is, certain object may belong to more than one 3er at the same 3me.
Measurement Method
Etalon for Func=onal Size Abran states that, • “The key concept of func3onality at the highest level of commonality that is present in all soEware was iden3fied as the ‘data movement’. This data movement concept was then assigned to the metrology concept of a size unit.” • The base unit of decomposi3on level is defined as the level of a single method within a single class. • The base unit of decomposi3on level is defined as Tier 0. Tier 0 is the level of decomposi3on in which further level of decomposi3on will be irrelevant of the measurement objec3ves of DMP method. (eg. Effort correla3on)
Conceptual Modeling
• We chose sequence diagram as the main soEware model for the empirical world to measure func3onal size alribute. As:
◦ Possible and automated round trip engineering. Code – Seq.Diagam & Seq.Diagram – Code. Many CASE tools and IDEs.
◦ Can be defined in each composi3on level. ◦ Are being developed for business level behavior and design level behavior. That is, they can be u3lized in both problem and solu3on domains.
◦ Incorporate more informa3on about a system than other UML diagrams. informa3on about the structures in a system as well as their rela3ons and system behavior.
Measurement Method
Opera=onal View for Func=onal Size
• In DMP measurement method, sequence diagram elements which represent the data movements are taken into account and other elements are used for suppor3ng informa3on.
• Sequence Diagram elements that represent data movement are: ◦ Synchronous message ◦ Asynchronous message ◦ Crea3on message ◦ Destruc3on message ◦ Self message ◦ Found message
Mapping Phase
Iden=fying Scenarios
Based on the decomposi3on level (Tier Value) for the measurement, defini3on of the func3onal processes and their triggering entries change.
1. The triggering entry of the func3onal process must be visible in the system model defined at this level.
2. The structural en3ty receiving the triggering entry must be visible in the system model defined at this level.
3. The output of the process (Exit or Write) must be visible in the system model defined at this level.
Measurement Tool
• Developed as a master project with Yalın Meriç supervised by Onur Demirörs, Erdir Ungan (2013) The automated XMI genera3on process was as below: 1. Select programming language. 2. Select folder where source code files are present. 3. Automa3cally generate all sequence diagrams for imported source
codes. 4. Create and save XMI file. 5. Count data movements for each method. 6. Assign objects to 3er level(s) 7. Enter 3er level 8. Consolidate DMs in that level and measure.
11 Data Movements
A1
A3
A2
B1 C1
C2
D1
D2
A1
A3
A2
B1 C1
C2
D1
D2
A
B
C
D
7 Data Movements
2 Data Movements
System
ValidaJon
• Theore3cal Valida3on
• Empirical Valida3on
ValidaJon
• Theore3cal Valida3on ◦ Rice Cooker Case ◦ Lavazza et al. and Levesque et al. separately measured the system with sequence diagrams.
◦ They reached different measurement conclusions. ◦ They modelled the func3onality differently.
No. Functional Process
Triggering Event Data Movement Description
Data Group DM CFP
1 Control Cooking Lamp
Tick (elapsed) Tick (elapsed) TimedEvents E 4
Cooking Mode Cooking Mode R
Cooking Status CookingLamp X
Cooking Status WarningLamp X
2 5 sec. signal management (control heater)
Every 5s signal Signal5 TimedEvents E 4 GetTargetTemp CookingState R ReadTemp TemperatureSens
or E
HeaterOn or
Heater command X
3 set target temperature
Signal30 Signal30 TimedEvents E 3
GetMode CookingMode R
SetTargetTemp CookingState W
4 Select Cooking Mode
Selected Cooking Mode
Selected Cooking Mode
E 2
Selected Cooking Mode
W
TOTAL 13
Rice Cooker COSMIC Measurement Results According to Levesque et. al.
No. Functional Process Triggering Event Data Movement Description
Data Group DM CFP
1 Control Cooking Lamp
Tick (elapsed) Tick (elapsed) TimedEvents E 2
On CookingLamp X
2 5 sec. signal management (control heater)
Signal5 Signal5 TimedEvents E 4 GetTargetTemp CookingState R ReadTemp TemperatureSensor E HeaterOn or
Heater command X
3 30 sec. signal management (set target temperature)
Signal30 Signal30 TimedEvents E 5
GetMode CookingMode R
Tick(elapsed) TimedEvents E
GetCookingTemp CookingSpecs R
SetTargetTemp CookingState W
TOTAL 11
Rice Cooker COSMIC Measurement Results According to Lavazza
• Levesque Iden3fied a func3onal process as “Select cooking mode”
• It complies with the defini3on of func3onal process.
• However, it is one decomposi3on level lower than the others.
• It should have been a sub process of “set target temperature”.
• That is : ◦ The size of the rice cooker is ◦ 11 DMP in Tier 1 ◦ 13 DMP in Tier 0
Emprical ValidaJon -‐ Findings
DMP had beOer correla=on with effort than COSMIC but worse than LOC
DMP vs. Effort R2 = 0,89
COSMIC vs. Effort R2 = 0,62
LOC vs. Effort R2 = 0,92
Example
• Upon receiving X from A, the system shall read X from B and compare their values. If X is greater than the Balance, system shall send Z to C. • COSMIC Size: E R X : 3
• Upon receiving the Amount_to_be_Withdrawn from the Customer, the system shall read Current_Balance from Customer's Account and compare their values.
• If the Amount_to_be_Withdrawn is greater, system shall send Warning SMS #213 to Customer's Cell Phone.
• COSMIC Size: 3
• Tier: 7
Example
• Upon receiving WebService_Withdrawal from System_ATM, the system shall read Current Balance from Customer's Account and compare their values.
• If the Amount_to_be_Withdrawn is greater, system shall send WebService_BalanceNotEnough to CRM.Module.
• COSMIC Size: 3
• Tier: 5
• Upon receiving WebService_Withdrawal from System_ATM, the system shall WebService.CustomerAccount.
Upon response, system will read Current Balance from the response and compare with The Amount_to_be_Withdrawn.
If The Amount_to_be_Withdrawn is greater, system shall send WebService_BalanceNotEnough to CRM.Module.
• COSMIC Size: 5
• Tier: 4
Example
• Upon receiving CurrentHeat from Thermometer, the system shall read RequiredHeat from ReqHeatRegister and compare their values.
• If CurrentHeat is greater than RequiredHeat, system shall send TurnoffHeater Signal to the Heater.
• COSMIC Size: 3
Example
• If we are measuring the embedded soEware on a microcontroller level of measurement can well be Tier : 0
• If the system is comprised of a higher level soEware running on a computer and communicate with intelligent sensors and heater through certain interfaces: ◦ If the boundary is only the soEware running on the computer, and see sensors as black boxes: COSMIC Size: 3, Tier:0
◦ If the boundary includes the sensors and interfaces: COSMIC Size: 3, Tier: 3+
Example
ContribuJons
Streamlining the Measurement Concepts
• We defined a measurement approach based on a concept that is common in both problem and solu3on domains; data movement.
• This streamlines measurement of both project and product alributes in two domains.
• Improves conversion of units, es3ma3ons, approxima3ons and normaliza3on of several size defini3ons and values.
Improvements in Reliability of Measurement Results
• Backfiring concepts of FSM from the source code elimina3ng the reliability issues in Tier 0. (product measurement)
• We traverse through higher levels u3lizing metadata for the lowest level. Only point of human interpreta3on is in the genera3on of this metadata which relates lower level concepts to higher level ones.
• This mi3gates measurement errors as there is less room leE for interpreta3on and renders errors recoverable by fixing the metadata.
BeWer Benchmarking
• Having decomposi3on level incorporated into measurement results makes the scope and abstrac3on of the measured soEware product visible.
• Comparing measurement methods on the same level of decomposi3on will greatly improve the accuracy of benchmarking studies.
BeWer EsJmaJons
• Instead of using conversion factors between two different size measurements that are essen3ally unrelated, we define an es3ma3on method which rely on the same concepts that the measurement method does.
• Es3ma3ons become less prone to gaps between domains and project phases.
• By measuring in lower levels of decomposi3on, DMP method measures data manipula=ons which would otherwise be leE out in higher levels into measurement. This also increases the accuracy of es3ma3ons.
EsJmaJon Approach
• N – Op3mal Decomposi3on Level (Tier #)
• Expected Accuracy
BeWer Measurement of SoQware Changes
• By DMP method, it is possible to measure specifica3ons defined in levels lower than func3onal user requirements.
• This makes measuring lower level so\ware changes more accurate than exis3ng FSM methods.
• Iden3fying the 3er level of change requests also improves the change management processes as the scope and impact of the change can be beler analyzed.
Further Studies
• The same approach can be applied to system specifica3on level or business process defini3ons.
• Another PhD thesis is being conducted in our research group on sizing business process models based on func3onality. ◦ It is possible to extend the problem-‐solu3on flow to the level of business processes.
◦ This would be a great opportunity to build a streamlined measurement and es3ma3on framework spanning the whole soEware engineering process.