centre for hci design art-scene: modelling complex design trade-off spaces with i* neil maiden head...

Centre for HCI Design

ART-SCENE: Modelling Complex

Design Trade-off Spaces with i*

Neil Maiden

Head of Centre


OverviewOur Centre and our research

– Who and what are we

ART-SCENE– A research prototype for scenario-based requirements

engineering and trade-off decision-making

A pattern language for submarine manoeuvring– Research questions and approach– A pattern language modelled using i* formalism– Research questions revisited

Future research to evaluate ART-SCENE– Pattern language is baseline for scalable systems

engineering research


The Centre and its EnvironmentUniversity Research Centre

– Independent department (www-hcid.soi.city.ac.uk) in City’s School of Informatics (www.soi.city.ac.uk)

Objectives– Undertake world-class basic and applied research into

the design of complex systems of which people are a significant component

Staff/students– Six academic, 7 research staff, 10 PhD students, 1

administrator and 1 visiting fellow

Research income– £1.6m income from 11 new research projects started

since January 2000


SIMP Basics and PartnersSystems Integration for Major Projects

– Part of EPSRC’s Systems Integration Initiative– August 2000-August 2003, £800,000, 15 person-years

Academic partners– City: Neil Maiden and 2 research staff– UMIST: Alistair Sutcliffe and 2 research staff– QMW: Norman Fenton and 1 research staff

Industrial partners– BAE SYSTEMS: Very complex systems of systems– Kennedy-Carter: Allan Kennedy– Intellectual Capital Services: Philip M’Pherson

Produce general outcomes for systems engineering


Our Research ObjectiveDeliver new decision-making capabilities

– Make requirement-architecture trade-offs through scenario analyses

• Analyse platform behaviour with different architectures through scenario analyses to determine requirement compliance

– Improve on ad-hoc modelling and decision-making capabilities when engineering large systems

Key research question– Is it possible to assess conformance of different

platform/equipment configurations to whole-system requirements in the context of different operational scenarios using a pattern-driven approach

– Informed by innovative theories of knowledge reuse from analogical reasoning research?


Start by Defining PatternsSIMP defines a pattern as

– A reusable architecture (the solution) to a collection of interconnected requirements (the problem) in well-defined scenarios (the context)

– The architecture rationale in terms of decisions made to trade-off requirements to select the best-fit architecture solution

– The effectiveness of the solution architecture in the defined scenarios, expressed as measurable levels of requirements compliance

Therefore what we need is– Semantics for expressing the elements of the problem,

solution and context of a pattern, interconnections between these elements, and formal constraints on constituent knowledge of a pattern


Defining Pattern LanguagesSIMP defines a pattern language as

– A collection of patterns that, together, constitute an organisation’s systems engineering knowledge for a defined domain

– The space of possible architecture solutions that have been considered to meet interconnected requirements, and trade-offs in that space

– Associations between patterns in the pattern language, expressed as associations between different elements in 2 or more patterns

– Facts and rules about requirements, architectures and scenarios in the defined domain

– A glossary of terms about the elements of requirements, architectures and scenarios in the defined domain


ART-SCENEResearch software prototype

– Analysing Requirements Trade-offs with Scenario-driven Evaluations

– Developed to investigate our key research question

ART-SCENE is a partial prototype– Implement using Microsoft VISIO2000, Visual Basic,

Access, Visual InterDev, and Excel– Integrate with software tools for requirements

management and cost-effectiveness calculation

Develop and evaluate 2 versions during the project– First version due in March 2002– Incremental evaluation and implementation


What ART-SCENE Seeks…..

Y MMY

YM

scenarios

requirements

architectures

Scenario Analyser

scenarios

requirements

architectureC

omp

uted

outcomes

Requirement compliance

Scenariooutcome

Integrated modelling and computational environment

Y MMYY

M

A large research challenge that might fail...

Scenarios withCREWS-SAVRE

i* modellingwith REDEPEND

Architectureswith iUML

Patterns andother analyses Computed

outcomes


Scenarioanalysis

Scenarioanalysis

Scenarioanalysis

The ART-SCENE Architecture

ComputationEngine

Measures of system effectiveness

CopernicusReDepend

Scenarioauthoring

Scenarioelaboration

Scenariowalkthrough

Scenarioanalysis

Goals, tasks& trade-offs Customer

scenarios

Generatedscenarios

Generatedscenarios

New systemrequirements

Computedscenario outcomes

OCDiUML

Requirements structures

Goals

Archi-tecture

Patterns


REDEPEND Software PrototypeVISIO2000-based software prototype

i* Modellingpalette

Common i* goal structure

Soft goal contribution

links


Using Patterns in Trade-Off AnalysisKey research question

– Analyse platform behaviour with different architectures through scenario analyses to determine requirement compliance

architecture

requirements

Scenario to acquire

Y MMY

YM

Fit criteria

scenariosScenario to

analyse

Scenariooutcome

Test outcome against fit criteria

How architecture performs in terms of requirements

Performance data


Modelling Patterns Using i* SemanticsLink requirements and architectures

– Individual agent dependencies• Link system, sub-system and adjacent system agents in terms

of goal and soft goal dependencies

– Individual means-end links• Connect architecture components (tasks at the moment) to

system goals/soft goals

– Patterns of agent dependencies and means-end links• Set of agent dependencies and means-end links that

encapsulate the association of an architecture to a requirement

Define the architecture trade-off space– Negative contribution links between soft goals

• Definition of the design trade-off space in terms of soft goals


BAE SYSTEMS Pattern LanguageDevelop pattern language to model

– Knowledge of design trade-offs made about manoeuvring systems of in-service submarines

• Stable domain that affords extensive reuse

– Elicit from BAE SYSTEMS expert systems engineers

Investigate 3 research questions1. Can experts agree a set of categories of design trade-

off decisions that are reusable in systems engineering?2. Can experts articulate the essence of the decision

categories that distinguish them from other decisions?3. Can we model this essence using the I* formalism so

that the experts agree that it is a accurate representation?


Method for Pattern ModellingWorked with pairs of BAE SYSTEMS engineers

– Developed list of key trade-off decisions

Incremental modelling of patterns– Open-ended 3-hour elicitation sessions with engineers– First-cut modelling of each pattern with i*– Focused 2-hour presentations to change and improve i*

pattern model with engineers– Agree and sign-off i* pattern model

Further i* modelling of manoeuvring domain– Standard SD model of manoeuvring agents and their

dependencies– Standard SR model of manoeuvring soft goals and their

contribution links


Answering Our Research QuestionsWe posited 3 research questions

1. Can experts agree a set of categories of design trade-off decisions that are reusable in systems engineering?

• YES: Results produced a list of >10 basic decision categories

2. Can experts articulate the essence of the decision categories that distinguish them from other decisions?

• YES: Experts were able to verbalise and agree discriminating characteristics of each decision category

3. Can we model this essence using the i* formalism so that the experts agree that it is accurate representation?

• YES: Four comprehensive i* models of 4 patterns - Experts stated that i* captured most essential characteristics

• Apart from elements of the solution architecture– This implies the need for extensions to the semantics….


How To Improve the PatternsBetter representation of solution elements

– Architectures of agents, components and connections– Extend with or link to the iUML approach

Continuous as well as discrete solution spaces– Complex systems have continuous solution spaces

• Impossible to enumerate all solutions using a qualitative modelling approach

– A means of linking characteristics of this solution space with solution architecture and component elements

Attaching performance data to patterns– We can define requirement-architecture trade-offs– Performance data is complex in submarine design

• From complex, domain-specific simulation models… But….


ART-SCENE: Putting It All TogetherManoeuvring pattern language

– Provides the basis for assessing effectiveness of patterns in trade-off decision-making

– Integrate the ART-SCENE prototype and evaluate it using BAE SYSTEMS submarine manoeuvring and Eurocontrol’s air traffic management domain

• REDEPEND i* modelling software tool• CREWS-SAVRE scenario elaboration and generation• Architecture modelling using formal iUML approach

Basis for exploring research question– Is it possible to assess conformance of different

platform/equipment configurations to whole-system requirements in the context of different operational scenarios

centre for hci design art-scene: modelling complex design trade-off spaces with i* neil maiden head...

Documents