Upload File

chunking:)an)empirical)evaluaon) of)so7ware)architecture)(?)) · chunking:)an)empirical)evaluaon)...

  • Home
  • Documents
  • Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)
24
Chunking: An Empirical Evalua3on of So7ware Architecture (?) Rachana Koneru David M. Weiss Iowa State University [email protected] [email protected] 1 With participation by Audris Mockus, Jeff St. Clair Weiss MECS 29 Sep 2013

Upload: others

Post on 07-Jun-2020

2 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

Chunking:  An  Empirical  Evalua3on  of  So7ware  Architecture  (?)  

Rachana  Koneru      David  M.  Weiss          Iowa  State  University  [email protected]                  

[email protected]  

1

With participation by Audris Mockus, Jeff St. Clair

Weiss MECS 29 Sep 2013

Page 2: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

2 Weiss MECS 29 Sep 2013

Page 3: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

Topics  

•  What  is  architecture?  

‑  Structures  ‑  So7ware  architecture  

v  Important  structures  

•  What’s  a  chunk?  

‑  Rela3onship  to  architecture  

•  What  data  do  we  need  to  find  chunks?  

•  What  algorithm  do  we  use?  

•  What  are  the  results?  

•  What’s  next?  

3 Weiss MECS 29 Sep 2013

Page 4: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

What is Software Architecture?  

•  Literally Hundreds of definitions#http://www.sei.cmu.edu/architecture/definitions.html#

•  Architecture is focused on #‑  Par33oning  the  whole  into  parts  ‑  Specifying  the  rela3ons  among  the  parts  ‑  Sa3sfying  Requirements  

Functional Requirements#End User Features …#

Other Engineering Requirements#Performance & Scalability, Reliability & Availability …#

4 Weiss MECS 29 Sep 2013

Page 5: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

5

The software architecture of a program or computing system is the structure or structures of the system, which comprise software elements, the externally visible properties of those elements, and the relationships among them.## "Externally visible” properties refers to those assumptions other elements can make of an element, such as its provided services, performance characteristics, fault handling, shared resource usage, and so on. #

Software Architecture in Practice (2nd edition), #(Bass, Clements, Kazman; Addison-Wesley 2003#

Weiss MECS 29 Sep 2013

Page 6: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

Structure (View)  

6

•  A structure is a binary relation#Set of ordered pairs {(a,b), (c,d), (b,a), (c,e)}##

•  Defining a structure#Define the set of elements#

a,b,c,d,e#Define the relation#

Enumeration#Rule##

•  Example: connected graph#Elements: nodes in the graph: a,b,c,d,e,f,g#Relation: “is connected to”#

{(a,b), (a,c), (b,d), (b,f), (f,e), (c.g)}#

a b c

d

e

f g

Weiss MECS 29 Sep 2013

Page 7: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

Key Software Architectural Structures  •  Module Structure!-  Explains the decisions used to design the information hiding structure of

the architecture, and shows how responsibilities are allocated among the major modules.#

-  What changes can be easily made?#o  How stable is my system with respect to change?#

•  Uses Structure#-  Describes the allowed “uses” relationships between modules and limits

what other modules the implementer of a module may use. #-  In what order should I implement?#

•  Process Structure#-  Defines the distinct processes in the system; Specifies the module(s)

that include the process, synchronization between processes …#-  What will the performance be?#

7 Weiss MECS 29 Sep 2013

Page 8: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

FWS  Module  Hierarchy  Graphical  View  

 

8

Sensor Device

FWS

Behavior Hiding Device Interface Software Design Hiding

Transmitter Device

Message Generation

Message Format

Averager Data Banker

Sensor Monitor

Weiss MECS 29 Sep 2013

Page 9: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

9 Weiss MECS 29 Sep 2013

Page 10: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

Chunks  

10 Weiss MECS 29 Sep 2013

Page 11: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

What’s  A  Chunk?  

•  A  chunk  is  a  set  of  code  that  has  the  property  that  a  change  that  touches  that  set  of  code  touches  only  that  set  of  code.  –  A>empt  to  iden@fy  informa@on  hiding  modules  empirically  in  exis@ng  code  –  Observe  the  effects  that  changes  have  on  the  sustainability  of  a  system  –  Try  to  find  good  models  of  design,  empirically  

•  Use  change  data  from  soJware  repositories  to  iden@fy  chunks  –  Many  caveats  and  constraints:  very  few  perfect  chunks  –  Difficult  to  find  good  data    –  An  early  result:  Mockus,  Weiss;  Globaliza@on  By  Chunking,  IEEE  SoJware,  

March/April  2001  

11 Weiss MECS 29 Sep 2013

Page 12: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

56 Weiss Industrial Measurement Dec 2011

Change Hierarchy

Weiss MECS 29 Sep 2013 12

Page 13: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

56 Weiss Industrial Measurement Dec 2011

Change Hierarchy

Weiss MECS 29 Sep 2013 13

Page 14: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

56 Weiss Industrial Measurement Dec 2011

Change Hierarchy

Weiss MECS 29 Sep 2013 14

Page 15: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

Caveats  and  Constraints  aka  “Op3miza3on  Criteria”  

•  Eliminate  “one-­‐3me”  chunks  –  Chunks  defined  by  one  change  

•  Eliminate  “super”  chunks  –  Chunks  that  include  changes  touching  20  files  or  more  

o   Complete  system  o   Large  subsystems  o  “20”  files  is  based  on  experimenta3on  with  the  data  

•  Consider  only  single-­‐purpose  changes,  such  as  bug  fixes  

•  Percentage  coupling:    The  percentage  of  MRs  in  the  chunk  that  touch  only  files  within  that  chunk.  –   A  measure  of  how  3ghtly  coupled  the  files  within  a  chunk  are.    The  

higher  the  coupling,  the  beZer  the  chunk.  A  perfect  chunk  has  100%    coupling.  

15 Weiss MECS 29 Sep 2013

Page 16: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

Algorithm  for  Finding  Chunks  (1)  1.   Randomly  pick  an  MR  that  touches  at  least  five  files,  but  no  more  than  20  files,  

from  the  en@re  set  of  MRs  in  the  dataset  obtained  from  the  project’s  change  data.  (Guarantees  a  minimum  chunk  size  that  is  large  enough  to  eliminate  one-­‐@me  changes.)  The  set  of  files  touched  by  this  MR  form  the  ini@al  chunk,  which  is  also  the  current  candidate  chunk.  

2.   Find  a  set  of  all  MRs  that  touch  each  of  the  files  in  the  current  candidate  chunk.    

3.   Find  a  set  of  all  files  touched  by  the  MRs  in  step  3.    

4.   Add  a  file  from  the  set  of  files  in  step  4  to  the  ini@al  chunk  only  if  it  improves  the  op@miza@on  criterion;  this  forms  the  current  candidate  chunk.    

5.   Repeat  step  5  un@l  all  the  files  in  the  set  of  files  in  step  4  are  considered.    

The  preceding  steps  ensure  that  the  search  space  for  all  file  combina@ons  is  sequen@ally  explored  so  that  poten@al  chunks  are  not  omi>ed.  This  also  leaves  out  false  chunks  resul@ng  from  combining  isolated  one-­‐@me  changes,  and  large  chunks  resul@ng  from  changes  that  touch  all  or  many  files.  

16 Weiss MECS 29 Sep 2013

Page 17: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

Algorithm  for  Finding  Chunks  (2)  6.   Repeat  steps  3  to  6  as  long  as  adding  a  file  improves  the  op@miza@on  criteria.  

7.   Repeat  steps  1  to  7  if  none  of  the  files  improve  the  op@miza@on  criteria  un@l  all  MRs  in  step  3  are  considered.    

8.   Repeat  steps  1  through  8  for  a  specified  @me  interval,  aJer  which  the  best  chunk  candidate  based  on  the  best  op@miza@on  criteria  is  generated.  

9.   Remove  the  files  cons@tu@ng  the  chunk  as  well  as  all  MRs  within  the  chunk  generated  in  step  9  from  the  ini@al  set  of  files  and  MRs  considered  respec@vely,  and  repeat  steps  1  to  9  to  generate  the  next  independent  chunk.  This  ensures  that  a  file  belongs  to  at  most  one  chunk.  

10.   Remove  the  files  cons@tu@ng  the  chunk  as  well  as  all  MRs  within  the  chunk  generated  in  step  9  from  the  ini@al  set  of  files  and  MRs  considered  respec@vely,  and  repeat  steps  1  to  9  to  generate  the  next  independent  chunk.  This  ensures  that  a  file  belongs  to  at  most  one  chunk.  

17 Weiss MECS 29 Sep 2013

Page 18: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

Applica3on  To  Three  Separate  Systems  Moodle Eclipse Europa Company X

Number of bug fixes

6016 MRs 9949 MRs

Type of project Open source, provides support for course management, more than 10 years old

Open source, provides support for software development, more than 10 years old

Large, long-standing telecommunications system, more than 15 years old

Language PHP Java C MR Data interval 4 years 18 months 6 years Number of MRs considered

6016 11,042 9949

Algorithm run time 8,000 seconds 4,000 seconds 10,000 seconds Minimum % Coupling Used

80% 80% 70%

18 Weiss MECS 29 Sep 2013

Page 19: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

Moodle  Analysis  

•  Organized  the  data  into  3  datasets  -  Moodle:  All  data  for  4  years  -  Moodle-­‐A:  First  2  years  -  Moodle-­‐B:  Second  2  years  

19

74  76  78  80  82  84  86  88  90  92  94  96  

1   2   3   4   5   6  

Percen

tage  Cou

pling  

Chunks  

Moodle   Moodle-­‐A   Moodle-­‐B  

Percentage Coupling of Top 6 Chunks for All Moodle Datasets

Weiss MECS 29 Sep 2013

Page 20: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

Eclipse  Analysis  

•  Organized  the  data  into  3  datasets  -  Europa:  All  data  for  18  months  -  Europa-­‐A:  First  9  months  -  Europa-­‐B:  Second  9  months  

20

Percentage Coupling of Top 11 Chunks for All Europa Datasets

0  10  20  30  40  50  60  70  80  90  100  

1   2   3   4   5   6   7   8   9   10   11  

Percen

tage  Cou

pling  

Chunks  

Europa   Europa-­‐A   Europa-­‐B  

Weiss MECS 29 Sep 2013

Page 21: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

Company-­‐X  Analysis  

•  Organized  the  data  into  3  datasets  -  Company-­‐X:  All  data  for  6  years  -  Company-­‐X  A:  First  3  years  -  Company-­‐X  B:  Second  3  years  

21

Percentage Coupling of Top 2 Chunks for All Company X Datasets

0  

20  

40  

60  

80  

100  

120  

1   2  

Percen

tage  Cou

pling  

Chunks  

Company-­‐X   Company-­‐X-­‐A   Company-­‐X-­‐B  

Weiss MECS 29 Sep 2013

Page 22: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

A  Few  Observa3ons  •  Chunks  can  generally  be  mapped  to  components  by  iden3fying  the  file  

system  directories  in  which  their  code  resides  -  Eclipse:  Very  clear  mapping  of  chunks  to  single  directories  

o  A  consequence  of  the  use  of  Java?  

-  Moodle:  One  to  many  mapping  of  chunks  to  directories  o  A  consequence  of  the  use  of  PHP?  

-  Company-­‐X:  Mapping  informa3on  of  chunks  to  directories  not  available    •  No  correla3on  of  chunk  size  with  chunk  percentage  coupling  •  Can  occasionally  observe  merging  of  chunks  and  increased  stability  of  

chunks  (increase  in  percentage  coupling  over  3me)  •  No  current  way  to  match  chunks  against  as-­‐designed  module  structure    

•  Valida3on  is  difficult  

-  Some  manual  inspec3on  of  a  (random)  sample  of  MRs  is  required  -  Manual  inspec3on  of  each  chunk  to  ensure  that  every  file  has  been  changed  

together  with  at  least  one  other  file  within  the  chunk    •  Rela3vely  few  chunks  iden3fied  

  22 Weiss MECS 29 Sep 2013

Page 23: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

Percentage  Coupling  and  Size  of  Chunks  for  Moodle  

23

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10

Chunks

Percentage Coupling Size Poly. (Size)

Weiss MECS 29 Sep 2013

Page 24: Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) · Chunking:)An)Empirical)Evaluaon) of)So7ware)Architecture)(?)) Rachana)Koneru)))David)M.)Weiss)))))IowaState)University)

What  Else  Would  We  Like  to  See?  

•  Replica3on  of  results  –  Same  data  set  and  different  data  sets  

•  Evolu3on  of  chunks  with  3me  –  When  is  it  3me  to  redesign?  –  Effects  of  experience  –  Effects  of  distribu3on  of  development  –  Time  and  effort  to  make  a  change  

•  Improved  chunk  detec3on  algorithm  

•  Iden3fica3on  of  good  designs  based  on  chunk  stability  

24 Weiss MECS 29 Sep 2013


Chunking and Named Entities

NP chunking in Hungarian

CHUNKING: What do I study?

Hierarchial chunking in sentence processing

Chunking Team Pattern Recog Icis2003

An OCaml definion of OCaml evaluaon, or,

CHUNKING: What do I study?. CHUNKING IS NOT “FLASH CARDS”

Chunking: Shallow Parsing

Multiplication By Chunking

chunking for subtraction€¦ · chunking for subtraction (1) When you break subtraction into chunks, it is easy to subtract the parts and put them back together. Use chunking to

Chunking and storyboarding

Dynamic chunking of component-authored information

Tagging and Chunking Best Practices

Text Chunking using NLTK

Memory Chunking An Elephant

Forcelandia 2016 PK Chunking

IIRecogniDon,&Characterizaon,&&&&&& Risk&Evaluaon ,&Risk

Chunking Content with Confidence

Chunking, Summary, & Annotation. Reading Strategies Chunking Summarization Annotation Hint: They all work together!!!!

Chunking. Chunking (psychology), a short-term memory mechanism and techniques to exploit itChunking (psychology) Chunking (writing), a method of splitting

Division By Chunking

From%self*evaluaon%to% metarepresentaon :% two ...joelleproust.org/wp-content/uploads/2012/09/Proustshorter_Louisvill… · From%self*evaluaon%to% metarepresentaon :% two%representaonal%systems?%

Chunking slides

February 2007CSA3050: Tagging III and Chunking 1 CSA2050: Natural Language Processing Tagging 3 and Chunking Transformation Based Tagging Chunking

Hindi Parts-of-Speech Tagging & Chunking

SunriseorSunset:ExploringtheDesignSpaceofBigData So7ware ...web.cse.ohio-state.edu/~lu.932/hpbdc2017/slides/hpbdc17-dk.pdf · NetworkBased(CompuAng(Laboratory (HPBDC‘17Panel( 9

Basics of Content Chunking

Chunking - School of Informatics, University of Edinburgh · Chunking in NLTK-Lite Chunking in Cass Chunking as Tagging Summary and Reading Problems with Full Parsing, 2 I Speed:

Whereare! you?!...EvaluatingWebsites… • Notanew%idea • Website%Evaluaon%Checklist:% • Accuracy,%Authority,%Objec4vity,%Currency,%Coverage% • Limitaons%of%website%evaluaon

DIY Evaluaon - UNM Evaluation Labevallab.unm.edu/for-organizations/2017-diy-eval-pres-2.pdf · DIY Evaluaon Annual Evaluaon Lab Workshop Melissa Binder, Director, NM Evaluaon Lab

Content Chunking & New Revenue Streams

Chunking, Annotation, & Summary. Reading Strategies Chunking Summarization Annotation Hint: They all work together!!!!

Algorithms for Minimum Risk Chunking

HDF5 Chunking and Compression - STAR · Why HDF5 chunking? • Chunking is required for several HDF5 features - Expanding/shrinking dataset dimensions and adding/”deleting” data

Applying Chunking Theory in Organizational Password Guidelinesjiito.informingscience.org/articles/JIITOv1p097-113Carstens30.pdf · Applying Chunking Theory in Organizational Password