(c) 2005 václav rajlich 1 software evolution and agile development václav rajlich department of...
TRANSCRIPT
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 11
Software Evolution and Software Evolution and Agile DevelopmentAgile Development
Václav RajlichVáclav RajlichDepartment of Computer ScienceDepartment of Computer Science
Wayne State UniversityWayne State UniversityDetroit, Michigan 48202, U.S.ADetroit, Michigan 48202, U.S.A
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 22
Confusing situationConfusing situation
• New recommendations how to develop New recommendations how to develop software software – Rational Unified ProcessRational Unified Process– Extreme ProgrammingExtreme Programming– SCRUMSCRUM– ……
• They often contradict what has been They often contradict what has been considered good software engineeringconsidered good software engineering– how to make a sense of this? how to make a sense of this?
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 33
ParadigmParadigm
• Thomas S. KuhnThomas S. Kuhn– ““The Structure of Scientific Revolutions”The Structure of Scientific Revolutions”
• ““Coherent tradition of scientific research” Coherent tradition of scientific research” – includes knowledge, techniques, research includes knowledge, techniques, research
agenda, culture of the field… agenda, culture of the field… – (more profound meaning than the usage (more profound meaning than the usage
common in software engineering) common in software engineering)
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 44
Paradigm changeParadigm change
• Discontinuity in the development of the Discontinuity in the development of the disciplinediscipline
• Kuhn collected extensive historical data on Kuhn collected extensive historical data on paradigm changeparadigm change– old paradigm cannot explain compelling new factsold paradigm cannot explain compelling new facts– new paradigm is a genuine revolution new paradigm is a genuine revolution – paradigm change is often contentious and paradigm change is often contentious and
traumatictraumatic– many examplesmany examples– phlogiston - > oxygen in 1770’sphlogiston - > oxygen in 1770’s
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 55
Resistance to paradigm Resistance to paradigm changechange• Knowledge that was accumulated up to that Knowledge that was accumulated up to that
point may lose its significance point may lose its significance – some knowledge may be completely lostsome knowledge may be completely lost
• Advantages of the new paradigm in disputeAdvantages of the new paradigm in dispute– attempts are made to extend old paradigm to attempts are made to extend old paradigm to
accommodate new factsaccommodate new facts– band-aid approaches try to fix old paradigmband-aid approaches try to fix old paradigm
• Final victory of the new paradigm Final victory of the new paradigm guaranteed by a generation changeguaranteed by a generation change
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 66
My claimMy claim
• Software evolution researchSoftware evolution research
• Agile + iterative processes Agile + iterative processes
• Represent paradigm change!Represent paradigm change!– Third Third paradigm change in 50 years of software paradigm change in 50 years of software
engineeringengineering
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 77
Paradigm change of 1950’sParadigm change of 1950’s
• Original programmers recruited from the ranks Original programmers recruited from the ranks of hardware engineers and mathematicians of hardware engineers and mathematicians – used ad-hoc techniques from their former fields used ad-hoc techniques from their former fields
• Software separated from the hardware in Software separated from the hardware in 1950’s1950’s– emerged as a distinct technologyemerged as a distinct technology– became independent productbecame independent product
• Programming became a new disciplineProgramming became a new discipline– developed a specialized set of skillsdeveloped a specialized set of skills
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 88
Paradigm change of 1970’sParadigm change of 1970’s
• Previous techniques did not scale up Previous techniques did not scale up – Brooks: “Mythical Man-Month” Brooks: “Mythical Man-Month” – demands of the new operating system OS/360 demands of the new operating system OS/360
taxed the limits of the programmers, project taxed the limits of the programmers, project managers, and the resources of the IBM managers, and the resources of the IBM corporationcorporation
• Paradigm change established discipline of Paradigm change established discipline of software engineering software engineering – dealt with complexity of softwaredealt with complexity of software– next step goes beyond programmingnext step goes beyond programming– introduced the waterfall metaphor introduced the waterfall metaphor
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 99
Waterfall metaphorWaterfall metaphor
• Used in construction and manufacturingUsed in construction and manufacturing– collect the requirements collect the requirements – create a design create a design – follow the design during the entire constructionfollow the design during the entire construction– transfer finished product to the user transfer finished product to the user – solve residual problems through maintenancesolve residual problems through maintenance
• Intuitively appealing metaphorIntuitively appealing metaphor– good design avoids the expensive late reworkgood design avoids the expensive late rework– waterfall became the dominant paradigm waterfall became the dominant paradigm
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 1010
Problems of waterfall Problems of waterfall
• Requirements volatility (Cusumano and Requirements volatility (Cusumano and Selby) Selby) – 30% or more requirements may change 30% or more requirements may change duringduring
development development – this is the direct result of the team’s learning this is the direct result of the team’s learning
process and software interoperability process and software interoperability
• Maintenance phase is not uniformMaintenance phase is not uniform– Lehner, PigoskiLehner, Pigoski– frequency of the changes in long lived systems frequency of the changes in long lived systems
peaks, then declinespeaks, then declines
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 1111
Standish group reportStandish group report
• In 1995In 1995– 31% of all software projects were cancelled31% of all software projects were cancelled– 53% were “challenged” (completed only with 53% were “challenged” (completed only with
great difficulty, with large cost or time great difficulty, with large cost or time overruns)overruns)
– only 16% could be called successful. only 16% could be called successful.
• Obviously, the waterfall metaphor did not Obviously, the waterfall metaphor did not solve the problems of software solve the problems of software development development
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 1212
Anticipation of changesAnticipation of changes
• If changes can be anticipated at design If changes can be anticipated at design time, they can be controlled by a time, they can be controlled by a parameterization, encapsulations, etc.parameterization, encapsulations, etc.– waterfall model still can be used waterfall model still can be used
• Experience confirms:Experience confirms:– many changes are not anticipated by the many changes are not anticipated by the
original designers original designers – inability to change software quickly and reliably inability to change software quickly and reliably
means that business opportunities are lostmeans that business opportunities are lost– only a band-aid solutiononly a band-aid solution
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 1313
PrototypingPrototyping
• Another attempt to extend waterfallAnother attempt to extend waterfall
• Create a prototype to capture Create a prototype to capture requirementsrequirements
• Problem: volatility continues after Problem: volatility continues after prototype has been completedprototype has been completed
• Another band-aid solutionAnother band-aid solution
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 1414
Paradigm change of 2000’sParadigm change of 2000’s
• New metaphor based on self-directed learningNew metaphor based on self-directed learning– Learner controls the processLearner controls the process
• Self-directed learning can be planned only to Self-directed learning can be planned only to a limited degreea limited degree– Needs a happy middle between rigid plan and Needs a happy middle between rigid plan and
chaoschaos– One semester at a timeOne semester at a time
• Iteration is the analogy of a semesterIteration is the analogy of a semester– Ends with a clear feedback (grade)Ends with a clear feedback (grade)– Continuation is based on that feedbackContinuation is based on that feedback
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 1515
Instances of the new Instances of the new paradigmparadigm• New life-cycle models emphasize software New life-cycle models emphasize software
evolution evolution – Staged model of software lifecycleStaged model of software lifecycle– Based on data from long-lived systems Based on data from long-lived systems
• Iterative developmentIterative development– Rational Unified ProcessRational Unified Process
• Agile developmentAgile development– SCRUMSCRUM– Extreme programmingExtreme programming
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 1616
Initial development
Evolution
first running version evolution changes
Servicing
code decay servicing patches
Close-down
Phase-out
servicing discontinued
switch-off
Staged model Staged model
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 1717
New research agendaNew research agenda
• Focus on the evolutionFocus on the evolution– Modifications in the existing software Modifications in the existing software
• Incremental change Incremental change – adds a new functionality or a new properties to adds a new functionality or a new properties to
the existing softwarethe existing software
• RefactoringRefactoring– cleans up software structure without changing cleans up software structure without changing
functionalityfunctionality
• Redocumentation, reengineering, Redocumentation, reengineering, migration, …migration, …
• Exciting and rich agendaExciting and rich agenda
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 1818
Incremental change (IC)Incremental change (IC)
• IC was supposed to happen only rarely IC was supposed to happen only rarely under the old paradigmunder the old paradigm
• IC was largely ignored by the researchers IC was largely ignored by the researchers and educators and educators – elephant in the living roomelephant in the living room
• The current programmers still must learn The current programmers still must learn IC on their own IC on their own – No presence in textbooks, curriculaNo presence in textbooks, curricula
• Large backlog of research issuesLarge backlog of research issues
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 1919
IC minicycleIC minicycle
• InitiationInitiation– New feature request from users, programmers, New feature request from users, programmers,
managers, marketing, …managers, marketing, …– Problem reportProblem report
• DesignDesign• ImplementationImplementation• ValidationValidation• ReleaseRelease
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 2020
IC designIC design
Concept
Software
Impact Set
Impact Analysis
Impact Analysis
Concept Location
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 2121
Concept locationConcept location
• Concept location finds where a change is Concept location finds where a change is to be madeto be made
• Change requests are most often Change requests are most often formulated in terms of domain conceptsformulated in terms of domain concepts– Example: “Correct error that arises when Example: “Correct error that arises when
trying to paste a text” trying to paste a text” – the programmer must find in the code the the programmer must find in the code the
locations where concept “paste” is locatedlocations where concept “paste” is located– this is the start of the changethis is the start of the change
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 2222
Concept location Concept location assumptionsassumptions• The programmer understands domain The programmer understands domain
concepts better than the codeconcepts better than the code– The knowledge of domain concepts is based The knowledge of domain concepts is based
on program use and is easier to acquireon program use and is easier to acquire
• All domain concepts map onto the All domain concepts map onto the fragments of the codefragments of the code– finding that fragment is concept locationfinding that fragment is concept location
– ““software engineer’s query”software engineer’s query”
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 2323
Concept location Concept location methodologies methodologies • Human knowledgeHuman knowledge• Traceability toolsTraceability tools• Dynamic search (execution traces)Dynamic search (execution traces)• Static search Static search
– "grep" (pattern matching)"grep" (pattern matching)– Search of dependency graphSearch of dependency graph– Information retrieval techniquesInformation retrieval techniques– ……
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 2424
Dependency Graph Search Dependency Graph Search • Programmer follows the dependencies Programmer follows the dependencies
among the modules.among the modules.
• Local functionalityLocal functionality– All concepts that are actually All concepts that are actually
implemented in the module and are not implemented in the module and are not delegated to others.delegated to others.
• Composite functionalityComposite functionality– All concepts of the module combined All concepts of the module combined
with all its supporting modules.with all its supporting modules.
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 2525
Search ProcessSearch Process• The variant of the depth first searchThe variant of the depth first search
• Programmer investigates the module's local Programmer investigates the module's local functionality. functionality.
• If the concept is not present in the local If the concept is not present in the local functionality, but is present in the composite functionality, but is present in the composite functionality, programmer continues search functionality, programmer continues search with one of the supporting modules.with one of the supporting modules.
• In case of wrong choice programmer In case of wrong choice programmer backtracks and begins the search with backtracks and begins the search with another supporting module.another supporting module.
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 2626
SearchSearch
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 2727
Information Retrieval Information Retrieval TechniqueTechnique• Latent Semantic Indexing (LSI)Latent Semantic Indexing (LSI)
– Preprocessing of the source code and Preprocessing of the source code and documentation documentation
– Executing of queries formulated in natural Executing of queries formulated in natural languagelanguage
– Retrieving and analyzing the resultsRetrieving and analyzing the results– Results are returned as a list, ranked by Results are returned as a list, ranked by
relevance (similarity) to the search queryrelevance (similarity) to the search query
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 2828
Set of concepts is not fixed Set of concepts is not fixed
• New concepts emerge during evolutionNew concepts emerge during evolution– set of concepts changes during the lifecycleset of concepts changes during the lifecycle– difficulty for traceability toolsdifficulty for traceability tools
• Origin of the conceptsOrigin of the concepts– Domain conceptsDomain concepts
• familiar to an end user: "credit card payment"familiar to an end user: "credit card payment"
– Design conceptsDesign concepts• "iterator pattern”"iterator pattern”
– Coding CONCEPTS Coding CONCEPTS • "network error while validating credit card”"network error while validating credit card”
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 2929
Primitive and latent Primitive and latent concepts concepts • Primitive concepts have to be enrichedPrimitive concepts have to be enriched
– Point-of-Sale: introduce credit card Point-of-Sale: introduce credit card paymentpayment
– old code: payment represented as just one old code: payment represented as just one numbernumber
• Latent concepts have to be implementedLatent concepts have to be implemented
– Student Registration: introduce prerequisite checkStudent Registration: introduce prerequisite check
– old code assumption: prerequisites are satisfiedold code assumption: prerequisites are satisfied
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 3030
Impact analysisImpact analysis
• Assesses full extent of changeAssesses full extent of change– Impact setImpact set
• Selects different implementation Selects different implementation alternativesalternatives
• Programmers often underestimate impact Programmers often underestimate impact setset
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 3131
Change implementationChange implementation
• Prepare software for the changePrepare software for the change– Opportunistic refactoring, prefactoringOpportunistic refactoring, prefactoring
• ActualizationActualization– Implementing new functionalityImplementing new functionality
• IncorporationIncorporation– Plug-in the new into the oldPlug-in the new into the old
• Change propagationChange propagation
• Improving software based on new knowledgeImproving software based on new knowledge– Postfactoring: Bad smells -> changePostfactoring: Bad smells -> change
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 3232
Change propagationChange propagation
• Visit components one-by-oneVisit components one-by-one
• If the visited component is modified, it may If the visited component is modified, it may no longer fitno longer fit– secondary changes must be made in interacting secondary changes must be made in interacting
(“neighboring”) components(“neighboring”) components
– secondary changes may trigger additional secondary changes may trigger additional changes (“ripple effect”) changes (“ripple effect”)
• Software is inconsistent during propagation Software is inconsistent during propagation
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 3333
Model of change Model of change propagationpropagation• Let C be a set of classesLet C be a set of classes• For b,c For b,c C, b C, b c, c,
{b,c} is interaction{b,c} is interaction<b,c> is mark <b,c> is mark b was visited (changed), c will be visited b was visited (changed), c will be visited
• Evolving interoperation graph (eig) is Evolving interoperation graph (eig) is a set E of interoperations and marks a set E of interoperations and marks <b,c> <b,c> E implies {b,c} E implies {b,c} E. E.
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 3434
Neighborhood Neighborhood
• All interactions, incoming, outgoing All interactions, incoming, outgoing marksmarks
G(b) = { {b,c} |G(b) = { {b,c} |c {b,c} c {b,c} E} interactions E} interactions
MM(b) = { <c,b> |(b) = { <c,b> |c <c,b> c <c,b> E} incoming E} incoming
M(b) = { <b,c> |M(b) = { <b,c> |c <b,c> c <b,c> E} outgoing E} outgoing
E(b) = G(b) E(b) = G(b) M(b) M(b) MM(b) (b)
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 3535
Visit Visit
• Couple of eigs <E,E'>Couple of eigs <E,E'>
E' = (E - E(b)) E' = (E - E(b)) E'(b') E'(b')
E'(b') = G'(b') E'(b') = G'(b') M'(b') M'(b') MM'(b') '(b')
• Marks point away from b’Marks point away from b’
M'(b') M'(b') { <b',c> | { <b',c> | c {b',c} c {b',c} E'(b') }E'(b') }
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 3636
Model of propagation Model of propagation sequencesequence
• Sequence of eigs Sequence of eigs EE11, E, E22, … E, … Enn
• Starts and ends with unmarked graphsStarts and ends with unmarked graphsmarked graphs in the middlemarked graphs in the middle
• <E<Eii, E, Ei+1i+1> is either> is either– a visita visit– or a backtrack: or a backtrack:
for some k, 0 < k < i, Efor some k, 0 < k < i, Ei+1i+1 = E = Ekk
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 3737
FinalityFinality
• Non-final propagation Non-final propagation – after visiting b and c, there may be an after visiting b and c, there may be an
immediate need to revisit b again:immediate need to revisit b again:
M'(b') M'(b') {<b',c>| {<b',c>| c {b',c} c {b',c} E'(b')} E'(b')}
• Final propagationFinal propagation M'(b') M'(b') {<b',c>| {<b',c>| c {b',c} c {b',c} E'(b')} - E'(b')} -
{<b',d>| {<b',d>| d <d,b> d <d,b> MM(b)}(b)}
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 3838
Case studyCase study
• Subject: application framework Drawlets Subject: application framework Drawlets
– adds graphical display to a host applicationadds graphical display to a host application
• Drawing canvas Drawing canvas
– lines, free-hand lines, rectangles, rounded lines, free-hand lines, rectangles, rounded
rectangles, triangles, pentagons, polygons, rectangles, triangles, pentagons, polygons,
ellipses, text boxes, imagesellipses, text boxes, images
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 3939
DrawletsDrawlets
• More than 100 classes, 35 interfaces and More than 100 classes, 35 interfaces and
40,000 lines of code40,000 lines of code
• Originally implemented in Smalltalk by Originally implemented in Smalltalk by
Kent Beck, Ward CunninghamKent Beck, Ward Cunningham– later ported into Javalater ported into Java
– ““perfect API” perfect API”
• Applet SimpleApplet runs in a browserApplet SimpleApplet runs in a browser
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 4040
SimpleApplet windowSimpleApplet window
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 4141
Top classes Top classes
SequenceOfFigures
ShapeTool
EllipseToolRectangleTool RectangularCreationTool PG_RectImageTool
Locator
CanvasTool
AbstractFigure
SimpleDrawingCanvas
DrawingCanvas
ToolPalette
SimpleApplet
ToolBar
LocatorConnectionHandle
PrototypeConstructionTool
SelectionTool ConstructionTool
StylePalette
LabelTool
Figure
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 4242
Change RequestChange Request
• Implement an "owner" for each figureImplement an "owner" for each figure– owner put that figure onto the canvasowner put that figure onto the canvas– only owner is allowed to move that figureonly owner is allowed to move that figure– each session declares a session ownereach session declares a session owner– this session owner will own all new figures this session owner will own all new figures
createdcreated– no other owner will be allowed to manipulate no other owner will be allowed to manipulate
themthem
• This change will make SimpleAppletThis change will make SimpleApplet more more versatile and usefulversatile and useful– support for cooperative worksupport for cooperative work
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 4343
ConceptConcept
• The concepts relevant to the changeThe concepts relevant to the change
– figure owner and session ownerfigure owner and session owner
• Both concepts are latentBoth concepts are latent
– old code assumes that there is just one owner old code assumes that there is just one owner
who owns both sessions and figureswho owns both sessions and figures
– look for “similar” concepts in the codelook for “similar” concepts in the code
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 4444
Location Location
SequenceOfFigures
ShapeTool
EllipseToolRectangleTool RectangularCreationTool PG_RectImageTool
Locator
CanvasTool
AbstractFigure
SimpleDrawingCanvas
DrawingCanvas
ToolPalette
SimpleApplet
ToolBar
LocatorConnectionHandle
PrototypeConstructionTool
SelectionTool ConstructionTool
StylePalette
LabelTool
Figure
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 4545
Actualization Actualization
SequenceOfFigures
ShapeTool
EllipseToolRectangleTool RectangularCreationTool PG_RectImageTool
Locator
CanvasTool
AbstractFigure
SimpleDrawingCanvas
DrawingCanvas
ToolPalette
SimpleApplet
ToolBar
LocatorConnectionHandle
PrototypeConstructionTool
SelectionTool ConstructionTool
StylePalette
OwnerIdentity
SimpleListener
LabelTool
Figure
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 4646
Incorporation Incorporation
SequenceOfFigures
ShapeTool
EllipseToolRectangleTool RectangularCreationTool PG_RectImageTool
Locator
CanvasTool
AbstractFigure
SimpleDrawingCanvas
DrawingCanvas
ToolPalette
SimpleApplet
ToolBar
LocatorConnectionHandle
PrototypeConstructionTool
SelectionTool ConstructionTool
StylePalette
OwnerIdentity
SimpleListener
LabelTool
Figure
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 4747
Propagation Propagation
SequenceOfFigures
ShapeTool
EllipseToolRectangleTool RectangularCreationTool PG_RectImageTool
Locator
CanvasTool
AbstractFigure
SimpleDrawingCanvas
DrawingCanvas
ToolPalette
SimpleApplet
ToolBar
LocatorConnectionHandle
PrototypeConstructionTool
SelectionTool ConstructionTool
StylePalette
OwnerIdentity
SimpleListener
LabelTool
Figure
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 4848
Propagation Propagation
SequenceOfFigures
ShapeTool
EllipseToolRectangleTool RectangularCreationTool PG_RectImageTool
Locator
CanvasTool
AbstractFigure
SimpleDrawingCanvas
DrawingCanvas
ToolPalette
SimpleApplet
ToolBar
LocatorConnectionHandle
PrototypeConstructionTool
SelectionTool ConstructionTool
StylePalette
OwnerIdentity
SimpleListener
LabelTool
Figure
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 4949
Propagation Propagation
SequenceOfFigures
ShapeTool
EllipseToolRectangleTool RectangularCreationTool PG_RectImageTool
Locator
CanvasTool
AbstractFigure
SimpleDrawingCanvas
DrawingCanvas
ToolPalette
SimpleApplet
ToolBar
LocatorConnectionHandle
PrototypeConstructionTool
SelectionTool ConstructionTool
StylePalette
OwnerIdentity
SimpleListener
LabelTool
Figure
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 5050
Propagation Propagation
SequenceOfFigures
ShapeTool
EllipseToolRectangleTool RectangularCreationTool PG_RectImageTool
Locator
CanvasTool
AbstractFigure
SimpleDrawingCanvas
DrawingCanvas
ToolPalette
SimpleApplet
ToolBar
LocatorConnectionHandle
PrototypeConstructionTool
SelectionTool ConstructionTool
StylePalette
OwnerIdentity
SimpleListener
LabelTool
Figure
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 5151
Propagation Propagation
SequenceOfFigures
ShapeTool
EllipseToolRectangleTool RectangularCreationTool PG_RectImageTool
Locator
CanvasTool
AbstractFigure
SimpleDrawingCanvas
DrawingCanvas
ToolPalette
SimpleApplet
ToolBar
LocatorConnectionHandle
PrototypeConstructionTool
SelectionTool ConstructionTool
StylePalette
OwnerIdentity
SimpleListener
LabelTool
Figure
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 5252
Propagation Propagation
SequenceOfFigures
ShapeTool
EllipseToolRectangleTool RectangularCreationTool PG_RectImageTool
Locator
CanvasTool
AbstractFigure
SimpleDrawingCanvas
DrawingCanvas
ToolPalette
SimpleApplet
ToolBar
LocatorConnectionHandle
PrototypeConstructionTool
SelectionTool ConstructionTool
StylePalette
OwnerIdentity
SimpleListener
LabelTool
Figure
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 5353
Propagation Propagation
SequenceOfFigures
ShapeTool
EllipseToolRectangleTool RectangularCreationTool PG_RectImageTool
Locator
CanvasTool
AbstractFigure
SimpleDrawingCanvas
DrawingCanvas
ToolPalette
SimpleApplet
ToolBar
LocatorConnectionHandle
PrototypeConstructionTool
SelectionTool ConstructionTool
StylePalette
OwnerIdentity
SimpleListener
LabelTool
Figure
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 5454
RefactoringRefactoring
• Modifies the structure, not functionalityModifies the structure, not functionality– after repeated incremental changes, the after repeated incremental changes, the
architecture become disorganized architecture become disorganized – refactoring re-introduces the orderrefactoring re-introduces the order
• Many different refactoring transformationsMany different refactoring transformations– simple renaming of an entity of the programsimple renaming of an entity of the program– refactoring a base classrefactoring a base class– refactoring design patterns, …refactoring design patterns, …
• Numerous articles, books, refactoring toolsNumerous articles, books, refactoring tools– many software environments have some many software environments have some
refactoring transformationsrefactoring transformations
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 5555
Shortening change Shortening change propagation propagation • Each class that creates new figures has Each class that creates new figures has
the function the function basicNewFigure(...)basicNewFigure(...) – two main parts:two main parts:– Create a new figure Create a new figure – If the figure was created at wrong location, If the figure was created at wrong location,
move it.move it.
• We refactored a new method called We refactored a new method called moveFigure(...)moveFigure(...)– made it a member of the base class made it a member of the base class ConstructionToolConstructionTool
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 5656
RefactoringRefactoring
SequenceOfFigures
ShapeTool
EllipseToolRectangleTool RectangularCreationTool PG_RectImageTool
Locator
CanvasTool
AbstractFigure
SimpleDrawingCanvas
DrawingCanvas
ToolPalette
SimpleApplet
ToolBar
LocatorConnectionHandle
PrototypeConstructionTool
SelectionTool ConstructionTool
StylePalette
LabelTool
Figure
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 5757
Splitting the rolesSplitting the roles
• Another technique that shortens change Another technique that shortens change propagationpropagation
• The same method used in two different rolesThe same method used in two different roles– the same code can do both jobsthe same code can do both jobs
• The propagating change highlights the The propagating change highlights the differencesdifferences– only one of the roles needs to be updated only one of the roles needs to be updated
– the other one can stay unchangedthe other one can stay unchanged
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 5858
Splitting the rolesSplitting the roles
• function function move(...)move(...) in in AbstractFigure AbstractFigure used in two waysused in two ways– to move the figure as requested by the userto move the figure as requested by the user
• must check user identitymust check user identity
– as a part of the new figure creationas a part of the new figure creation• does not need to check user identitydoes not need to check user identity
• We split We split move(...)move(...)and and secureMove(…)secureMove(…)
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 5959
Code decay Code decay
• Causes transition from evolution to servicing Causes transition from evolution to servicing
• Large changes in the functionality are no Large changes in the functionality are no longer possiblelonger possible
• Substantially lowers the value of the software Substantially lowers the value of the software – software managers should be aware of this software managers should be aware of this
phenomenonphenomenon– it can happen by accidentit can happen by accident– very little research into what causes code decay very little research into what causes code decay
and how to prevent itand how to prevent it
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 6060
ServicingServicing
• Changes are limited to patches and Changes are limited to patches and wrapperswrappers– less costly, but they cause further less costly, but they cause further
deterioration deterioration
• Process is very different from Process is very different from evolutionevolution– no need for senior engineersno need for senior engineers
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 6161
From servicing back to From servicing back to evolutionevolution
• in the current practice: in the current practice: – very hard, very rarevery hard, very rare
• the knowledge has to be rebuiltthe knowledge has to be rebuilt
• for all practical reasons, the for all practical reasons, the transition from evolution to servicing transition from evolution to servicing is irreversibleis irreversible
• worthy research goalworthy research goal
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 6262
Close-downClose-down
• The software use is disconnected The software use is disconnected – What is the current life of successful What is the current life of successful
softwaresoftware
• Cost of changing to another system Cost of changing to another system • What to do with long-lived data?What to do with long-lived data?
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 6363
Cognitive aspectsCognitive aspects
• The software evolution requires the The software evolution requires the programmers to learnprogrammers to learn– This learning will determine the success of the This learning will determine the success of the
projectproject
• New programmers must absorb project New programmers must absorb project knowledgeknowledge– Current documentation systems fail to capture Current documentation systems fail to capture
this knowledge adequatelythis knowledge adequately– entrance of the new programmers into an old entrance of the new programmers into an old
project is difficult. project is difficult.
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 6464
ConjectureConjecture
• Code decay is in fact the situation where Code decay is in fact the situation where complexity of the code overwhelms the complexity of the code overwhelms the cognitive capabilities of the programmerscognitive capabilities of the programmers
• Positive feedback between Positive feedback between – loss of structureloss of structure– knowledge of the systemknowledge of the system
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 6565
Knowledge constructionKnowledge construction
• AbsorptionAbsorption– the new facts are added to the existing the new facts are added to the existing
knowledgeknowledge • DenialDenial
– the new facts are rejectedthe new facts are rejected • ReorganizationReorganization
– learners reorganize their knowledge to aid learners reorganize their knowledge to aid absorption of new factsabsorption of new facts
• ExpulsionExpulsion– part of the knowledge becomes obsolete or part of the knowledge becomes obsolete or
provably incorrect and the learners reject itprovably incorrect and the learners reject it
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 6666
Self-Directed LearningSelf-Directed Learning
BloomBloom AbsorptionAbsorption DenialDenial ReorganizationReorganization ExpulsionExpulsion
RecognitionRecognition √√ √√ √√ √√
ComprehensionComprehension √√ √√ √√ √√
ApplicationApplication √√ √√ √√ √√
AnalysisAnalysis √√ √√ √√ √√
SynthesisSynthesis √√ √√ √√ √√
EvaluationEvaluation √√ √√ √√ √√
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 6767
Early resultsEarly results
• Experienced programmers use Experienced programmers use reorganization and expulsion more oftenreorganization and expulsion more often– Inexperienced programmers keep things the Inexperienced programmers keep things the
way they areway they are
• Experienced programmers spend more Experienced programmers spend more time clarifying domain concepts before time clarifying domain concepts before codingcoding
• Top-down comprehension uses higher Top-down comprehension uses higher Bloom’s levels than bottom-upBloom’s levels than bottom-up
• Aim: recording project knowledgeAim: recording project knowledge
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 6868
ConclusionConclusion
• The old waterfall paradigm tried to freeze The old waterfall paradigm tried to freeze requirements for the duration of software requirements for the duration of software development development – led to too many project failuresled to too many project failures
• The new paradigm emphasizes software The new paradigm emphasizes software evolutionevolution– embrace the new paradigm!embrace the new paradigm!– long backlog of research problems long backlog of research problems – exciting topics for software engineering researchexciting topics for software engineering research– these topics have been neglected by the these topics have been neglected by the
researchers researchers
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 6969
ReferencesReferences
1. Beck, K. 1. Beck, K. Extreme Programming ExplainedExtreme Programming Explained. Addison Wesley, Reading, . Addison Wesley, Reading, MA, 2000.MA, 2000.
2. Brooks, F.P. 2. Brooks, F.P. The Mythical Man-MonthThe Mythical Man-Month. Addison-Wesley, Reading, MA, . Addison-Wesley, Reading, MA, 1982.1982.
3. Cusumano, A.M. and Selby, R.W. How Microsoft Builds Software. 3. Cusumano, A.M. and Selby, R.W. How Microsoft Builds Software. Communications of ACMCommunications of ACM (June 1997). 53-61. (June 1997). 53-61.
4. Eick, S.G., Graves, T.L., Karr, A., Marron, J.S. and Mockus, A. Does 4. Eick, S.G., Graves, T.L., Karr, A., Marron, J.S. and Mockus, A. Does Code Decay? Assessing the Evidence from Change Management Code Decay? Assessing the Evidence from Change Management Data. Data. IEEE Transactions on Software EngineeringIEEE Transactions on Software Engineering, , 2727 (1) (2001), 1-12. (1) (2001), 1-12.
5. Fowler, M. 5. Fowler, M. Refactoring: Improving the Design of Existing CodeRefactoring: Improving the Design of Existing Code. . Addison Wesley, Reading, MA, 1999.Addison Wesley, Reading, MA, 1999.
6. Jacobson, I., Booch, G. and Rumbaugh, J. 6. Jacobson, I., Booch, G. and Rumbaugh, J. The Unified Software The Unified Software Development ProcessDevelopment Process. Addison Wesley, 1999.. Addison Wesley, 1999.
7. Johnson, J.H. Micro Projects Cause Constant Change, The Standish 7. Johnson, J.H. Micro Projects Cause Constant Change, The Standish Group International, Inc, 1997.Group International, Inc, 1997.
8. Kuhn, T.S. 8. Kuhn, T.S. The Structure of Scientific RevolutionsThe Structure of Scientific Revolutions. The University of . The University of Chicago Press, Chicago, 1996.Chicago Press, Chicago, 1996.
(C) 2005 Václav Rajlich(C) 2005 Václav Rajlich 7070
9. Rajlich, V. and Bennett, K. A Staged Model for the Software 9. Rajlich, V. and Bennett, K. A Staged Model for the Software Lifecycle. Lifecycle. ComputerComputer, , 3333 (7) (July 2000) 66-71. (7) (July 2000) 66-71.
10. Rajlich, V., Gosavi, P. Incremental Change in Object-Oriented 10. Rajlich, V., Gosavi, P. Incremental Change in Object-Oriented Programming. Programming. IEEE SoftwareIEEE Software, , 2121 (July/August 2004). 62 - 69. (July/August 2004). 62 - 69.
11. Rajlich, V., Xu, S., Analogy of Incremental Program Development 11. Rajlich, V., Xu, S., Analogy of Incremental Program Development and Constructivist Learning. In and Constructivist Learning. In Second IEEE Int. Conf. On Cognitive Second IEEE Int. Conf. On Cognitive InformaticsInformatics, (2003), IEEE Computer Society Press, 98 - 105., (2003), IEEE Computer Society Press, 98 - 105.
12. Wilde, N., Buckellew, M., Page, H., Rajlich, V. and Pounds, L. A 12. Wilde, N., Buckellew, M., Page, H., Rajlich, V. and Pounds, L. A Comparison of Methods for Locating Features in Legacy Software. Comparison of Methods for Locating Features in Legacy Software. Journal of Systems and SoftwareJournal of Systems and Software, , 6565 (2) (February 2003), 105-114. (2) (February 2003), 105-114.
13. A. Marcus, V. Rajlich, J. Buchta, M. Petrenko, A. Sergeyev, Static 13. A. Marcus, V. Rajlich, J. Buchta, M. Petrenko, A. Sergeyev, Static Techniques for Concept Location in Object-Oriented Code, IEEE Techniques for Concept Location in Object-Oriented Code, IEEE International Workshop on Program Comprehension, 2005, 33 - 42International Workshop on Program Comprehension, 2005, 33 - 42
14. V. Rajlich, Changing Paradigm of Software Engineering, to be 14. V. Rajlich, Changing Paradigm of Software Engineering, to be published in Communications of ACMpublished in Communications of ACM