1 karl reed univ. balzano feb. 2006 some issues in the engineering of widely deployed software...

1

Karl Reed univ. balzanoFeb. 2006Some issues in the engineering of

widely deployed software intensive systems-dynamic functional variation and its interpretation(strictly work in progress…)

Department of Computer Science & Computer Engineering, La Trobe University

by Karl Reed, and Luo,X

2

Karl Reed univ. balzanoFeb. 2006

1. We have a system consisting of a collection of interacting (statically or dynamically) bound components in which either a single component or sub-system can be replaced with another…

2. If the functionality represented by the replaced “part” is changed,

A/ What does this mean to the user?

B/ How shall we (or should we) control the functionality-variation at the user level?

C/ What kinds of functional variation can we identify?

D/ What mechanisms can be provided to allow this to be propagated to the user level?

(Normally I’d be arguing that we need systems stability, and that this is a measure of quality???)

The Problem..

3


1. We have a system consisting of a collection of interacting (statically or dynamically) bound components in which either a single component or sub-system can be replaced with another…

2. If the functionality represented by the replaced “part” is changed,

A/ What does this mean to the user?

B/ How shall we (or should we) control the functionality-variation at the user level?

C/ What kinds of functional variation can we identify?

D/ What mechanisms can be provided to allow this to be propagated to the user level?

(Normally I’d be arguing that we need systems stability, and that this is a measure of quality???)

The Problem..

4


If systems visible functionality changes If systems visible functionality changes unexpectedly, users will get a surpriseunexpectedly, users will get a surprise!!!!!!

If systems INVISIBLE functionality changes If systems INVISIBLE functionality changes unexpectedly, users will get a surprise, may be unexpectedly, users will get a surprise, may be a delayed traumatic a delayed traumatic shock !!shock !! … …

But, are surprises newBut, are surprises new!!!!!!

5


•Key to understanding this….Key to understanding this….

•Users often have “new” functions thrown at Users often have “new” functions thrown at them by complex systems..them by complex systems..

They are used to DVF already..They are used to DVF already..

•They cannot tell if the functionality was They cannot tell if the functionality was always there, or was added dynamically since always there, or was added dynamically since the last use..the last use..

””All” we need to do is to study these cases!!All” we need to do is to study these cases!!

6


David Harel said (1992) that may David Harel said (1992) that may be we already know the answers, be we already know the answers,

and need to study what we are and need to study what we are ready do (and know-KR)ready do (and know-KR)

7


“F1. Current software has too many surprises. The sources of surprise are poorly understood.”

Sources of surprises... - Real and apparent ambiguity in the means of representation of systems, e.i. Languages (cf 3 pages of c++ with 3 pages of government regulations)

- Real and apparent unpredictability in system behaviour

“F2. Key sources of software surprise include immature or poorly integrated software domain sciences, construction (product) principles, and engineering processes. Software research emphases have swung from process to product research, with weak coverage of domain sciences and integration.”

Surprise….!!!(nsf report on s/w research 1998)

8


“F1. Current software has too many surprises. The sources of surprise are poorly understood.”

Sources of surprises... Real and apparent unpredictability in behaviour...

No surprises….!!!(nsf report on s/w research 1998)

“Teenagers have less trouble with PC software because they are adept at playing computer games” Charles Wright, editor Melbourne Age “green pages” computer section 2000

“Building ‘bots’ that play computer games with near human competence is not that hard” US researcher in AI….

9


1. Dynamically “varying” systems of (autonomous) dynamically linked components can lead to variant functional or non-functional behavioural variation.

2. If functional variation is allowed, what can user’s deal with? How to control it? What is both acceptable and allowable?

3. In terms of what is acceptable, we can make use of a number of properties of real systems (Shaw 1999) where the internal states/transitions are quite “fuzzy”, and where a user actually accepts a range of outcomes (hence, varying functionality). In fact, humans may not work with precise systems often.

-Also, humans work with large systems whose apparent ambiguity (Reed, 2000) can appear as functional variation

1. What is allowable .. An (functional) operational envelope could be defined in principal. Functionality out-side this envelope could be rejected (Balzer 2004)- BUT RECORDED, AND PRESENTED TO THE USER, WHO COULD ADD IT TO THEIR OPERATIONAL ENVELOPE (or a systems administrator could)

2. Pt.4. Constitutes a “controlled mutant adoption” process.

3. Pt.3. May involve a kind of reverse HCI,or a human-centred fault tolerant approach.

Agenda

10


what are they?

-A “surprise” (for our purposes) is some behaviour of a system’s which causes or could cause a user to make an error1, or excessive stress and discomfort in resolving the behaviour

-Occur inherently in Lehmann’s E-type systems

-Occur WHEN developers BELIEVE they are building E-type systems

Examples..introduction of new expense claim s/w suddenly impacts the work-loads, stress and financial security of 100’s of people

…no logical relationship between functions in s/w and semantics of menus they are in

..almost un-usable web-sites

..SCS system failures are better known

-INCLUDE some un-expected functionality (that requires some effort to interpret?)

WHAT ARE SURPRISES --- WHO KNOWSABOUT THEM AND WHAT CAN WE DO ABOUT THEM?

11


Who knows about them?

-The SCS community places great effort on identifying “unexpected behaviours”and controlling their impact.

-Those working on systems with adaptive behaviour and user-error recovery.

-Fault-tolerant community (already been mentioned)

-Games technologists (one of the Fruanhofer Institut’s has looked at this)

-Extreme Programmers and iterative developers “think” they are dealing with “surprises”

-Product-line strategists

-lateral thinkers (here, the “surprise” is an unrecognised “use” which allows functional substitution)

What can we do about them?

-Elevate their definition and management to a high-level design feature rather than an implementation problem to be avoided

-Those working on systems with adaptive behaviour and user-error recovery.

-Study the behaviour of people working in/with systems whose behaviour changes

-at the design level- deal with the adoption and control of autonomous functional variation

WHAT ARE SURPRISES --- WHO KNOWSABOUT THEM AND WHAT CAN WE DO ABOUT THEM?

12


Where functionality has been removed..

-Maybe analogous to a fault-tolerant situation (however, the system could actively seek a replacement component)

-User may agree that they can manage without this (operational envelope).

-User may seek-out a replacement function--rejecting the original

-Fault-tolerant community (already been mentioned)

-Games technologists (one of the Fruanhofer Institut’s has looked at this)

-”Grace-full degradation” of OS in the 70’s..

Where functionality has been changed

-Need to define “conformant” and “non-conformant” changes, e.g., a data representational change

-Need to recognise unacceptable (I.e. unadoptable at the system level) change (op-envelope issue)

-Can we define functionality extractors-correctors? (recognise and correct a functional miss-match?- a re-use - “glue-code” problem, contract violation?)

adoption of autonomous functional variation due to some type of component “change”

13


Formal approaches….

-Component-contract specifications specificy..

1. What semantic variation in service-component that can be tolerated

2. Semantic definition of service’s actual functionality

-Semantic reasoning about aggregated functionality, propogating the changes upwards until either they reaches the user, OR violate some semantic constraint.

Informal Approaches.. (may be an operational approach?)

-Examine examples of functional variation visible to users,

-Using these, develop rules for specifiying functional variations at the component level, and for their transmission upwards (an operational approach to the formal)

Suggest there is actually a lot of data and examples to examine..

e.g. behaviour of pc-desk top s/w aplications present to the user as having unpredictable changes in functionality

relationship between knowledge, experience,skill and tools is relevant

Dealing with changing function..

14


The word-processor example..(Cervantes,Humberto and Hall, R. S. (2004)- Cervantes et al. Reported an experiment where the functionality of a word processor

changed dynamically..- The spell-checker and grammar checkers were changed from one natural language to

another..

- However, the “word-processor” software is not really effected by such a functional change!!!

- The USER sees the functional variation?

- We can define a kind of DVF compliance for this case..

1. The spell-checker/grammar checker are for a language using roman characters

2. The language is written left to right (does this matter?)

- We can define this as a class of DVF agnosticsm---1. We can look for other systems which have this kind of DVF property

- What happens if language violates these requirements?

Kinds of Functional Change…

15


An Invisible DVF…

- An application uses a DB and the

DB is changed so that it….

1. No-longer stores some data field..

2. No-longer checks for consistency of data

3. Roll-Back and Recovery are turned off

4. Changed from relational to OO, or

from hierarchic to OO


Interpretation.

- May be detected at a later time

- May be detected at a much later time

- May be detected at a much later time

- May not be detected at all

16


Messaging System Subjected to DVFMessaging System Subjected to DVF

DB repository

UIS

CommandProcessor

User Pin

checker

Command

Sequence

Checker

MessageSystem

We change the functionality We change the functionality of DB so it copies all of DB so it copies all transactions to some other transactions to some other repository repository

call ms(“DBR”,..)call ms(“DBR”,..) target:= “CP”;target:= “CP”;

call ms(target,..)call ms(target,..)

read(target);read(target);

call ms(target,..)call ms(target,..)

17


Visible DVF a plausibility study…

- A Browser based query application, returns the age, office-location, phone number of an employee…

- DVF of the DB has the result that additional data (qualifications, home phone number) are returned.. and displayed in a new browser window..

User reaction.. “This is useful, I want this in future..”

- How could we deal with this?

1. User could move the window around, or the data items, to construct a new screen

2. Application generator could generate new scripts/code for the new screen

3. They could be compiled under Java/Corba/.NET if need be

4. Turned into a new application, incorporating the new functionality


18


Assumes mechanisms for adopting functional variation and propagating it through a design

An (functional) operational envelope could be defined in principal. Functionality out-side this envelope could be rejected- BUT RECORDED, AND PRESENTED TO THE USER, WHO COULD ADD IT TO THEIR OPERATIONAL ENVELOPE (or a systems administrator could)

Constitutes a “controlled mutant adoption” process.

May involve a kind of reverse HCI,or a human-centred fault tolerant approach.

Use approaches from security (although of conceptual value only)--perhaps consider form of intrusion (e.g. audit trail monitoring)

Adaptive user profile generation could be part of this

Needs..mechanisms for describing the added functionality to users and administrators

roll-back mechanisms

Balzer (2002).. Contained Execution next few slides

Operational Envelope approach…

19


Safe-EmailSafe-EmailAttachmentsAttachmentsDemo Demo

20


SafeEmail Attachments

M

M

M M

WrapperSafetyRulesk

AttachmentHandler

Spawn

• Wrapper encapsulateseach spawned process

SafeEmail Attachments

M

M

M M

WrapperSafetyRulesj

AttachmentHandler

• Each opened attachment spawns new process

SpawnSafeEmail Attachments

M

M

M M

WrapperSafetyRulesi

Attachment

Attachment

EmailClient

Safe EmailSafe EmailAttachmentsAttachments

GeneratorInterpreter

21

Karl Reed univ. balzanoFeb. 2006Wrapped execution-Balzer

22


File System

filefile

filefile

filefile

file

Read

RealFiles

file file

filefile

file

filefile

Virtual File System

VirtualFile

VirtualFile

VirtualFile

VirtualFile Virtual

File VirtualFile

VirtualFile

VirtualFile

Registry

Read

RealKeys

key key

keykey

key

keykey

keykey

keykey

key keykey

Virtual RegistryVirtual

KeyVirtual

Key

VirtualKey

VirtualKey Virtual

KeyVirtual

Key

VirtualKey

VirtualKey

COTSApplication

M

M

M M

Wrapper

Contained ExecutionContained Execution• Transparent

Redirection

Write Virtual Files

Write Virtual Keys

• Selectively Applied

• Once created virtual resource used instead of real resource

Demo

23


Contained ExecutionContained Execution

Like a Virtual MachineExecution is isolated

Unlike a Virtual MachineProcess-Level (instead of machine-level)Selective (instead of copying entire

environment)Incremental (copies created as needed)

24


1. Use of component contract violation exceptions…

a. If a component violates its contract, then an exception is generated, and new data either discarded, or, if in the form of attribute-value pairs (e.g. XML)

b. The DVF originator could also provide the exception handler??

c. What do we do if there is data missing (see b.)?

d. What do we do if semantics of data are changed (see b.)?

2. Generalised “extra” parameter in all component invocations

a. call varied( p1,p2,…pn,array_of_pointers,extra_data_flag, error_flag)

b. Require extra data in attribute-value pairs

3. General Parameter Miss-Matches (e.g. number, type)

a. See above…

4. Interpreting Functional variation..

a. See Katayama(2001) on evolution..

Dealing with DVF.. A Possible Mechanism..

(may not be original…)

25


1. Study the way humans work with large systems with apparent ambiguity

2. Develop the operational envelope approach..

3. Develop a “controlled mutant adoption” process.

Research Agenda (one component)

1 karl reed univ. balzano feb. 2006 some issues in the engineering of widely deployed software...

Documents