ibm rational © 2005 ibm corporation the complexity of programming models grady booch ibm fellow

91
IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

Upload: robert-robertson

Post on 26-Mar-2015

221 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation

The Complexity ofProgramming Models

Grady BoochIBM Fellow

Page 2: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation2

How much software exists in the world?

SLOC is a measure of labor (not of value)

– Old code never dies (you have to kill it)

– Some code is DOA

Some assumptions

– 1 SLOC = 1 semicolon

– Number of software professionals worldwide

– %of software professionals who cut code

– SLOC/developer/year

– $100US/SLOC

Page 3: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation3

Number of software professional worldwide

Number of IT professionals worldwide

y = -128.47x3 + 12800x

2 - 59294x + 146623

0

2,000,000

4,000,000

6,000,000

8,000,000

10,000,000

12,000,000

14,000,000

16,000,000

194519481951195419571960196319661969197219751978198119841987199019931996199920022005

Number of IT professionals worldwide(assumptions)

Poly. (Number of IT professionalsworldwide (assumptions))

Page 4: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation4

% of software professionals who cut code

% of IT professionals worldwide who cut code

y = -0.0075x + 0.7575

0%

10%

20%

30%

40%

50%

60%

70%

80%

1945194719491951195319551957195919611963196519671969197119731975197719791981198319851987198919911993199519971999200120032005

of IT professionals worldwide who cut code % )assumptions (

Poly . (% of IT professionals worldwide who cut))code (assumptions

Page 5: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation5

SLOC/developer/year

New or modified source lines of code per year per developer

y = -0.0328x3 + 4.8392x

2 - 67.596x + 1062.8

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

1945194719491951195319551957195919611963196519671969197119731975197719791981198319851987198919911993199519971999200120032005

New or modified source lines of code per year)per developer ( assumptions

Poly . (New or modified source lines of code per))year per developer ( assumptions

Page 6: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation6

New or modified SLOC/year and cumulative

New or modified source lines of code per year per developer & cumulative

0

100,000,000,000

200,000,000,000

300,000,000,000

400,000,000,000

500,000,000,000

600,000,000,000

700,000,000,000

800,000,000,000

194519481951195419571960196319661969197219751978198119841987199019931996199920022005

New or modified source lines of code peryear

Cumulative source lines of code

Page 7: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation7

Dimensions of software complexityHigher technical complexity - Embedded, real-time, distributed, fault-tolerant - Custom, unprecedented, architecture reengineering - High performance

Lower technical complexity - Mostly 4GL, or component-based - Application reengineering - Interactive performance

Highermanagement complexity - Large scale - Contractual - Many stake holders - “Projects”

Lowermanagement complexity - Small scale - Informal - Single stakeholder - “Products”

Defense MIS System

Defense Weapon SystemTelecom

Switch

CASE Tool

National Air TrafficControl System

Enterprise IS(Family of ISApplications)

CommercialCompiler

BusinessSpreadsheet

IS ApplicationDistributed Objects

(Order Entry)

Small ScientificSimulation

Large-ScaleOrganization/Entity

Simulation

An average software project - 5-10 people - 3-9 month duration - 3-5 external interfaces - Some unknowns & risks

EmbeddedAutomotive

Software

IS ApplicationGUI/RDB

(Order Entry)

Royce

Page 8: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation8

Stakeholders and views

A given system is many things to many different stakeholders– End user– Customer– Sys admin– Project manager– System engineer– Developer– Architect– Maintainer– Tester– Other systems

Multiple realities, multiple views and multiple blueprints exist

Page 9: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation9

Simplicity has different points of view

User

– Simple metaphors and gestures

End-user programmer

– Access to significant parts and flexible mechanisms for behavior

Architect

– Common architectural patterns

Developer

– Common design patterns and idioms

Tester

– Access to significant parts

Page 10: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation10

Simplicity has different points of view

Business analyst

– Clear separation of rules

Database analyst

– Purity of semantics

Security officer

– Clear and perfectly executed policies

Administrator

– Clear separation of components

Page 11: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation11

In the presence of essential complexity,establishing simplicity in one part of a system

requires trading off complexity in another

Page 12: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation12

Creating the illusion of simplicity

Page 13: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation13

Simplicity in languages

Tradeoff between primitiveness and convenience

– Control structures

Tradeoff between explicitness and abstraction

– Java garbage collection

Tradeoff between performance of development and performance of execution

– VisualBasic

– Smalltalk

Tradeoff between packaging for design versus packaging for development versus packaging for deployment

– Beans

– Services

– Aspects

Page 14: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation14

A programming model specifies thesemantic universe within which the developer labors

and is defined by the languages, platforms, tools,and best practices of that constellation

Page 15: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation15

Web-centric programming model

Languages

– HTML

– CSS

– XSL

– XML

– SQL

– RSS

– Java

– JavaScript

– PHP

– Flash

– UML

Platforms

– Linux

– Apache

– MySQL

– J2EE

Best practices

– Coding

– Design patterns

– Deployment

– User interface

– Accessibility

– Internationalization

– Security

– Logging

– Backup

Tools

– Eclipse

– Dreamweaver

– Photoshop

– ClearCase

– ClearQuest

– RSA

– Portfolio Manager

– RequisitePro

– Tester

Page 16: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation16

Alternative programming models

Game developer

High performance computing

Command and control

Artificial intelligence

Domain-specific frameworks

– …

Handbook of Software Architecture, http://www.booch.com/architecture

Page 17: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation17

A system is shaped by a myriad ofdesign decisions by different stakeholders

that work to balance the forcesswirling around the system

Page 18: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation18

Forces in civil architecture

Avoiding failure - Safety factors - Redundancy - Equilibrium

Compression

Load

Tension

Load

Kinds of loads - Dead loads - Live loads - Dynamic loads

Any time you depart from established practice, make ten times theeffort, ten times the investigation. Especially on a very large project. - LeMessuier

Page 19: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation19

Forces on software

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Page 20: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation20

Why is software inherently complex?

Complexity of the problem domain

Difficulty of managing the development process

Fluidity of software

Fundamental challenges of discrete systems

Page 21: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation21

Complexity of the problem domain

Volume of requirements

Presence of competing/contradictory requirements

Non-functional requirements that push the limits of software

Requirements churn

Difficulty of communicating requirements

Impedance mismatch among stakeholders

Unrestrained external complexity

Software drag

Page 22: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation22

The limits of software

Page 23: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation23

Difficulty of managing the development process

Software as a team sport

Presence of multiple languages, platforms, processes, architectures, and tools

Issues of scalability

Technology churn

Page 24: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation24

Scalability

Size up

– Increasing database size by a factor of x increases query response time by at most a factor of x.

Speed up

– Increasing the capacity of your hardware configuration by a factor of x decreases your query response time by no less than a factor of x.

Scale up

– Increasing the workload on your system by a factor of x while maintaining response time and/or throughput requires increasing your capacity by a factor of no more than x.

Scale out

– Increasing workers by a factor of x requires replicating your capacity by a factor of at most x.

http://www.intelligententerprise.com/db_area/archives/1999/991602/scalable.jhtml

Page 25: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation25

Fluidity of software

Software springs from pure thought and is intrinsically malleable, yet it can be made manifest in our hardware systems, limited only by our vision (and certain immutable laws of physics and software)

Page 26: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation26

Fundamental challenges of discrete systems

Non-continuous behavior of discrete systems

Combinatorial explosion of states

Corruption from unexpected external events

Lack of mathematical tools and intellectual capacity to model the behavior of large discrete systems

Page 27: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation27

Essential complexity

“Einstein argued that there must be simplified explanations of nature, because God is not capricious or arbitrary. No such faith comforts the software engineer. Much of the complexity that he must master is arbitrary complexity.” [Brooks]

We may master essential complexity, but we can never make it go away.

Page 28: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation28

Measuring complexity of biological systems (syntactic)

Kolmogorov

Entropy

Mean component size

Number of behaviors exhibited

Species richness, relative to tolerance to risk

Species guilds

Energy flow

Grammatical complexity

Number of feedback loops

Cyclomatic measures (arcs, vertices, and components)

Graph complexity

Hamming distance

http://www.carleton.ca/~hmasum/complex.html

Page 29: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation29

Measuring complexity of biological systems (semantic)

Wordcount of description

Minimal description length (Rissanen)

Measure of environmental knowledge

Evolvability

Eigenbasis/measure of survivable environmental states

Program complexity

http://www.carleton.ca/~hmasum/complex.html

Page 30: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation30

Measuring complexity of software-intensive systems

Kolmogorov

– Relative size of a program capable of generating a given string

Entropy

– Enumeration of states and transitions

http://cscs.umich.edu/~crshalizi/notebooks/complexity-measures.html

Page 31: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation31

Measuring simplicity

If we don’t know how to measure complexity, it is reasonable to suggest that we don’t know how to measure simplicity

“I can’t define it, but I know it when I see it.” [Supreme Court Justice Brennan]

Page 32: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation32

Beauty

Elegance is not an approach to finding a solution to a problem, it is the label we stick on the optimum solution

Elegance is doing the most with the least

Elegance means simplicity and less new code.

An elegant solution solves the whole problem." [Fisher, p. 37]

Fisher & Gipson, “In Search of Elegance”

Page 33: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation33

Triggers of complexity

Significant interactions

High number of parts and degrees of freedom

Nonlinearity

Broken symmetry

Nonholonomic constraints

– Localized transient anarchy

Flood, et al, Dealing with Complexity

Page 34: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation34

Attributes of a complex system

“Frequently, complexity takes the form of a hierarchy, whereby a complex system is composed of interrelated subsystems that have in turn their own subsystems, and so on, until some lowest level of elementary components is reached.” [Courtois]

Hierarchic systems are decomposable if they can be divided into identifiable parts; they are nearly decomposable if their parts are not completely independent. [Simon]

Page 35: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation35

Attributes of a complex system

The choice of what components in a system are primitive is relative arbitrary and is largely up to the discretion of the observer of the system.

As systems evolve, objects that we once considered complex become the primitive objects upon which more complex systems are built.

Page 36: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation36

Attributes of a complex system

Intracomponent linkages are generally stronger than intercomponent linkages. This fact has the effect of separating the high-frequency dynamics of the components – involving the internal structure of the components – from the low-frequency dynamics - involving interaction among components. [Simon]

Page 37: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation37

Attributes of a complex system

Hierarchic systems are usually composed of only a few different kinds of subsystems in various combinations and arrangements. [Simon]

Page 38: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation38

Decomposible and nearly-decomposible systems

Vertically, the components of a complex system tend to be organized in increasing layers of abstraction

Horizontally, the components of a complex system tend to be clustered according to frequency

Both vertically and horizontally, the most resilient systems tend to exhibit loose coupling and tight cohesion among components

Simon, The Organization of Complex Systems

Page 39: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation39

Components

Loosely-coupled components adapt more easily to change

Loosely-coupled components permit time- and space-separation of processing

Overall flexibility can be enhanced by limiting the number of different kinds of components in the system (the system’s alphabet)

Alphabets are necessary but insufficient

Complex systems also require common languages, defining semantics of structural organization of alphabetic elements and interactional behavior among structures

Simon, The Organization of Complex Systems

Page 40: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation40

Languages

Must have sufficient variety in its primitive processes so that no meaning is absolutely excluded from expression

Must have sufficient flexibility in its rules of combination so that any nuance can be expressed by building up composite structures

Simon, The Organization of Complex Systems

Page 41: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation41

Attributes of complex systems

Complex systems will evolve from simple systems much more rapidly if there are stable intermediate forms than if there are not. [Simon]

A complex system that works is invariable found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system. [Gall]

Page 42: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation42

Complex adaptive systems

Emergent behavior

Attributes

– Classification of components

– Identity of objects

– Non-linearity of behavior

– Flow of objects

– Diversity

– Use of internal models

– Clustering

Holland, Hidden Order

Page 43: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation43

Characteristics of self-organizing systems

Dynamic, requiring continual interactions among lower-level components to produce and maintain structure

Exhibit bifurcation leading to multistable systems

– Strange attractors

Sante Fe Institute

Page 44: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation44

Self-organization in biological systems

Pattern formation in slime molds

Feeding aggregation of bark beetles

Synchronized flashing among fireflies

Fish schooling

Nectar source selection by honey bees

Trail formation in ants

Swarm raids of army ants

Colony thermoregulation in honey bees

Comb patterns in honey bee colonies

Wall building by ants

Termite mount building

Construction Aagorithms by wasps

Dominance hierarchies in paper wasps

Camazine, et al, Self-Organization in Biological Systems

Page 45: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation45

Creating order in biological systems

Self-organization

– Emergence of patterns at the global level solely from numerous interactions among lower-level components of the system, the rules for which are executed using only local information

Imposed organization

– Direction from a supervisory leader

– Organic blueprints or recipes

– Patterns in the environment

Camazine, et al, Self-Organization in Biological Systems

Page 46: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation46

Complexity, Functionality, and Understandability

Complexity

Functionality

Understandability

Page 47: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation47

Fundamentals never go out of style

Page 48: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation48

Shearing layers of change

Site

Skin

Structure

Services

Space plan

Stuff

Brand, How Buildings Learn

Page 49: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation49

Fundamentals of well-engineered software-intensive systems

Crisp abstractions

Clear separation of concerns

Balanced distribution of responsibilities

Simplicity via common abstractions and mechanisms

Page 50: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation50

Abstraction

All abstractions are context-dependent

All non-trivial abstractions are, to some degree, leaky (and leaky abstractions are problematic). [Joel on Software]

There is no such thing as a perfect abstraction

– Perfect is the enemy of good enough

Page 51: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation51

Worse Is Better

Simplicity is the most important consideration in a design.Both implementation and interface must be simple, though it is more important for the implementation to be simple.

The design must be correct in all observable aspects; it is slightly better to be simple than correct.

The design must not be overly inconsistent; it is better to drop those parts of the design that deal with less common circumstances than to introduce implementational complexity.

The design must cover as many imporatant situations as practical; completeness can be sacrificed in favor of any other quality.

Gabrial, “Worse is Better”

Page 52: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation52

Loose abstractions

Over-engineering a solution is the most common approach to dealing with complexity, yet it typically leads to total implosion.

Software which is flexible, simple, sloppy, tolerant and altogether forgiving turns out to be most resilient. [Bosworth]

Page 53: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation53

The entire history of software engineeringIs one of rising levels of abstraction

Assembly -> Fortran/COBOL -> Simula -> C++ -> JavaNaked HW -> BIOS -> OS -> Middleware -> Domain-specificWaterfall -> Spiral -> Iterative -> AgileProcedural -> Object Oriented -> Service Oriented Early tools -> CLE -> IDE -> XDE -> CDEIndividual -> Workgroup -> Organization

Languages:Platforms:

Processes:Architecture:

Tools:Enablement:

Page 54: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation54

Attacking complexity

Fundamentals

– Crisp abstractions

– Clear separation of concerns

– Balanced distribution of responsibilities

– Simplicity via common abstractions and mechanisms

Relax a constraint

Make assumptions

Page 55: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation55

Architecture defined

Architecture n (1563)

– The art or science of building or constructing edifices of any kind for human use

– The action or process of building

– Architectural work; structure, building

– The special method of ‘style’ in accordance with which the details of the structure and ornamentation of a building are arranged

– Construction or structure generally

– The conceptual structure and overall logical organization of a computer or computer-based system from the point of view of its use or design; a particular realization of this

Oxford English Dictionary, 2nd ed.

Page 56: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation56

Physical systems

Mature physical systems have stable architectures– Aircraft, cars, and ships– Bridges and buildings

Such architectures have grown over long periods of time

– Trial-and-error– Reuse and refinement of proven solutions – Quantitative evaluation with analytical methods

Mature domains are dominated by engineering efforts

– Analytical engineering methods– New materials– New manufacturing processes

Page 57: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation57

Software-intensive system

A system in which software is the dominant, essential, and indispensable element

– E-commerce system

– IT (business) system

– Telephone switch

– Flight control system

– Real-time control system (e.g. industrial robot)

– Sophisticated weapons system

– Software development tools

– System software (e.g. operating systems or compilers)

Page 58: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation58

Architecting software is different

No equivalent laws of physics

Transparency

Complexity– Combinatorial explosion of state space– Non-continuous behavior– Systemic issues

Requirement and technology churn

Low replication and distribution costs

Page 59: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation59

Architecture defined

Software architecture is what software architects do

Beck

Page 60: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation60

Architecture defined

Perry and Wolf, 1992

– A set of architectural (or design) elements that have a particular form

Boehm et al., 1995

– A software system architecture comprises

• A collection of software and system components, connections, and constraints

• A collection of system stakeholders' need statements• A rationale which demonstrates that the components, connections, and

constraints define a system that, if implemented, would satisfy the collection of system stakeholders' need statements

Clements et al., 1997

–The software architecture of a program or computing system is the structure or structures of the system, which comprise software components, the externally visible properties of those components, and the relationships among them

http://www.sei.edu/architecture/definitions.html

Page 61: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation61

Common elements

Architecture defines major components

Architecture defines component relationships (structures) and interactions

Architecture omits content information about components that does not pertain to their interactions

Behavior of components is a part of architecture insofar as it can be discerned from the point of view of another component

Every system has an architecture (even a system composed of one component)

Architecture defines the rationale behind the components and the structure

Architecture definitions do not define what a component is

Architecture is not a single structure -- no single structure is the architecture

Page 62: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation62

Architecture defined

Architecture establishes the context for design and implementation

CODE

implementation

design

architecture

Architectural decisions are the most fundamental decisions; changing them will have significant ripple effects.

Page 63: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation63

Architecture defined

IEEE 1471-2000

– Software architecture is the fundamental organization of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution

Software architecture encompasses the set of significant decisions about the organization of a software system

– Selection of the structural elements and their interfaces by which a system is composed

– Behavior as specified in collaborations among those elements

– Composition of these structural and behavioral elements into larger subsystems

– Architectural style that guides this organization

Booch, Kruchten, Reitman, Bittner, and Shaw

Page 64: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation64

Architecture defined

Software architecture also involves

– Functionality

– Usability

– Resilience

– Performance

– Reuse

– Comprehensibility

– Economic and technology constraints and tradeoffs

– Aesthetic concerns

Page 65: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation65

Architectural style defined

Style is the classification of a system’s architecture according to those with similar patterns

A pattern is a common solution to a common problem; patterns may be classified as idioms, mechanisms, or frameworks

Page 66: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation66

Model, views, concerns, and stakeholders

A model is a simplification of reality, created in order to better understand the system being created; a semantically closed abstraction of a system

A view is a representation of a whole system from the perspective of a related set of concerns

A concern is those interests which pertain to the system's development, its operation or any other aspects that are critical or otherwise important to one or more stakeholders

A stakeholder is an individual, team, or organization (or classes thereof) with interests in, or concerns relative to, a system

Page 67: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation67

Stakeholders and views

Architecture is many things to many different stakeholders– End user– Customer– Sys admin– Project manager– System engineer– Developer– Architect– Maintainer– Tester– Other systems

Multiple realities, multiple views and multiple blueprints exist

Page 68: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation68

Representing software architecture

Logical View

End-user Functionality

Implementation View

Programmers Configuration management

Process View

PerformanceScalabilityThroughput

System integrators

Deployment View

System topologyCommunication

Provisioning

System engineering

Conceptual Physical

Use Case View

Clements, et al, Documenting Software Architectures

Page 69: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation69

Adapting views

Not all systems require all views

– Single process (ignore process view)

– Small program (ignore implementation view)

– Single processor (ignore deployment view)

Some systems require additional views

– Data view

– Security view

– Other aspects

Page 70: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation70

Logical view

The view of a system’s architecture that encompasses the vocabulary of the problem and solution space, the collaborations that realize the system’s use cases, the subsystems that provide the central layering and decomposition of the system, and the interfaces that are exposed by those subsystems and the system as a whole

Focuses on

– Functionality

– Key Abstractions

– Mechanisms

– Separation of concerns and distribution of responsibilities

Page 71: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation71

Process view

The view of a system’s architecture that encompasses the threads and processes that form the system’s concurrency and synchronization mechanisms

Focuses on

– Performance

– Scalability

– Throughput

Page 72: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation72

Implementation view

The view of a system's architecture that encompasses the components used to assemble and release the physical system

Focuses on

– Configuration management

Page 73: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation73

Deployment view

The view of a system’s architecture that encompasses the nodes that form the system’s hardware topology on which the system executes

Focuses on

– Distribution

– Communication

– Provisioning

Page 74: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation74

Use case view

The view of a system’s architecture that encompasses the use cases that describe the behavior of the system as seen by its end users and other external stakeholders

Page 75: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation75

Relations among views

Logical view Implementation view

Process view

Deployment view

Page 76: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation76

Architecture metamodel

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Page 77: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation77

Architecture metamodel

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Page 78: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation78

Architecture metamodel

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Page 79: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation79

The architecture of biological systems

Gene

Cell component

Cell

Tissue

Organ

System

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Page 80: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation80

Cross-cutting concerns in biological systems

Gene

– Reproduction

– Protein creation

Cell component (mitochondria)

– Metabolism

– Glutamate-mediated excitotic neurlogical injury

– Cellular proliferation

– Regulation of the cellular redox state

– Heme synthesis

– Heat production

Page 81: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation81

Cross-cutting concerns in biological systems

Cell

– Structure

– Metabolism

– Reproduction

– Protein synthesis

– Transport/container

Tissue

– Structure

– Work

– Transport/container

Page 82: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation82

Cross-cutting concerns in biological systems

Organ (liver)

– Digestion

– Carbohydrate metabolism

– Glucoenogenesis

– Glycogenesis

– Breakdown of insulin

– Lipid metabolism

– Cholesterol synthesis

– Production of triglycerides

– Coagulation factors

– Neutralization of various products

– Storage of glucose and various vitamins

– Red cell production for the fetus

System (circulatory)

– Transport

– Heat regulation

– Healing mechanism

Page 83: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation83

The reification of concerns

Concerns are not isomorphic to structure

In biological systems, these aspects evolved simultaneously and interdependently at each level of abstraction

– They existed a priori as reactions to evolutionary forces

– Post hoc we can name them

Page 84: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation84

Representing software architecture

Logical View

End-user Functionality

Implementation View

Programmers Configuration management

Process View

PerformanceScalabilityThroughput

System integrators

Deployment View

System topologyCommunication

Provisioning

System engineering

Conceptual Physical

Use Case View

Clements, et al, Documenting Software Architectures

Page 85: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation85

Cross-cutting concerns in software-intensive systems

Some structures and behaviors crosscut components• Security• Concurrency• Caching• Persistence

Such elements usually appear as small code fragments sprinkled throughout a system

Such elements are hard to localize using traditional approaches

Page 86: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation86

The role of aspect-oriented software development

Remember the fundamentals

– Crisp abstractions

– Clear separation of concerns

– Balanced distribution of responsibilities

– Simplicity via common abstractions and mechanisms

The current sweet spot for aspects involves elements of each of these fundamentals

– Especially with regard to building crisp abstractions and the separation of concerns for roles relative to packaging

– This impacts primarily the interplay of the logical view and the use case view

Page 87: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation87

What’s missing/what’s next

Remember the already complex programming model

– Don’t make it more complex by adding yet another orthogonal mechanism

The current pragmatic focus is upon transformation tools that focus on already visible artifacts

– The harder focus - plus the one that is most disruptive yet most potentially valuable - is upon transformation tools that focus on deep semantic representations and then the creation of these traditional artifacts by reflection

• I.e. source code as a pretty-printed side-effect, not a central object

Page 88: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation88

Summary

This stuff is fundamentally, wickedly hard

– And it’s not going to get any better in my lifetime

• And I plan on having a long life

Remember that the world doesn’t need Yet More Technology

– We need less

• And ultimately, the best technology is invisible

Page 89: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation89

Bibliography on complexity

Allen, T. & Starr, T., Hierarchy: Perspectives for Ecological Complexity, University of Chicago: 1982.

Axelrod, R., The Complexity of Cooperation, Princeton: 1997.Barrow, J., Davies, P., & Harper, C., Science and Ultimate Reality, Cambridge

University Press: 2004.Bowker, G. & Star, S., Sorting Things Out: Classification and its Consequences, MIT

Press: 1999.Buchanan, M., Nexus, Norton: 2002.Camazine, S., Deneubourg, J., Franks, N., Sneyd, J., Theraulaz, G., & Bonabeau,

E., Self-Organization in Biological Systems, Princeton: 2001.Duda, R., Pattern Classification, 2nd ed., Wiley: 2001.Epxtein, J. & Axtell, R., Growing Artificial Societies, MIT Press: 1996.Flood, R. & Carson, E., Dealing With Complexity: An Introduction to the Theory and

Application of Systems Science, Plenum Press: 1988.Gleick, J., Chaos: Making a New Science, Penguin Books: 1987.Hollland, J., Hidden Order, Perseus Books: 1995.Johnson, S., Emergence, Scribner: 2001.

Page 90: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation90

Biography on complexity

Kauffman, S., At Home in the Universe, Oxford University Press: 1995.Kipfer, B., The Order of Things, MJF Books: 2001.Lakoff, G., Women, Fire, and Dangerous Things: What Categories Reveal about the

Mind, University of Chicago: 1987.Lakoff, G. & Johnson, M., Metaphors We Live By, University of Chicago: 1980.Markman, E., Categorization and Naming in Children, MIT Press: 2002.Nicolis, G. & Prigogine, I., Exploring Complexity, Freeman: 1989.Pattee, H., Hierarchy Theory: The Challenge of Complex Systems, George Braziller:

1973.Prigogine, I., The End of Certainty, Free Press: 1996.Simon, H., The Sciences of the Artificial, 2nd ed., MIT Press: 1969.Waldrop, M., Complexity: The Emerging Science at the Edge of Order and Chaos,

Simon & Schuster: 1992.

Page 91: IBM Rational © 2005 IBM Corporation The Complexity of Programming Models Grady Booch IBM Fellow

IBM Rational

© 2005 IBM Corporation91

Thank you!Thank you!