Applicability of MDD to

Software Factories

The FAST Experience

ISP – Industrialisation of Software Production

1- Industrialisation Strategies

Industrialisation of Software Production

ISP as key expertise area for Tecnalia/ESI with more than 15 years of experience on ISP related proyects (14 FP7-EUREKA and 30 market references) Objective: To support an organisation to increase applications production capacity through a software development faster, cheaper and with a better quality through the industrialisation process of software production

There are several ISP

strategies, and there is no

one-size-fits-all technology

nor method that supports the

wide range of potential

scenarios. We group all

strategies in 4 groups.

Industrialisation of Software Production

Product Lines for product-centric reuse strategies

Software Factories for multi domain-centric reuse

Architecture Driven for reference architecture

constrained development

Community Driven for fuzzy and less structured

organisations

We select strategies based on characteristics such as reuse opportunities and risks analysis, current and desired reuse practices, technical context, productivity control and measurement and important issues of the organizational context such as company size, customer base, metrics of projects and products, etc.

ISP – Community Driven

Community Driven: GMC by Tecnalia

GMC Platform

GMCCrawler

Enterprise EcoSystem

GMCClones

GMC4Migration

GMCDeveloperProfile

GMC IDE

GMCLicenceAnalyst

GMCPeople

GMCDojo

ISP – Architecture Driven

Architecture Driven, examples:

GEMDE – MDE toolbox for different architectures

• MDE for RT Embedded systems with MARTE/SysML

• MDE for legacy modernisation with X2U and custom made DSLs

ISP – Product Line

Generic tool for the design, implementation and management of Software Product Lines following a MDSD (Model Driven Software Development) approach.

Version 3.0 available. Selected for the European CESAR RTP. CVL support.

Ready for verticalisations and joint delivery with ICT companies. FAST powered.

ISP – Software Factory – Next Generation

IIn

du

stri

ali

sati

on

Co

ntr

ol

& M

on

ito

r

Pro

du

ctiv

ity

to

ols

Pro

cess

Org

an

isa

tio

n

SO

A –

OS

Gi

Ecl

ipse

ID

E

2 – MDA, MDD, MDE, MDSD…

Model Driven X

Let’s not focus on the conceptual differences between MDA-MDD, MDE, MDSD..

Let’s focus quickly on basic ideas and value

Models Abstraction of an object/phenomena in the real world

Simplified view of the complex reality

Conforms to metamodels!

Metamodels Defines what a model looks like

A kind of model grammar (-meta)

A collection of concepts of a certain domain

Formal description ensuring information consistency

End-user

Developer

Sales and field support

Development manager

System administrator

Functionality

Architect

Ease of use

Performance and throughput

Reliability

Ease of customization

Price

Development costs, skills

On time delivery

Stability and maintainability

Ease of integration

Ease of diagnosing problems

Ease of introducing modifications

Testability and tractability

Structure and dependencies between parts

Ease of installation

Complexity

Functionality

Technology

Skills

Process

Organization and culture

Qualities

Business Concerns

FOR COMMUNICATION…

FOR INTEGRATION…

Analyze

Requirem.

Design

Validate

& Verify

Build Code

Requirements Manag.

Code GenerationModel CheckingCASE Tool

Performance SimulationSchedulability Analysis WCET AnalysisReliability Analysis

Common

Repository

FOR MULTIDISCIPLINARY APPROACHES…

Domain in Embedded

Systems Development

Ab

str

act

ion

Le

ve

l

So

ftw

are

Ele

ctr

on

ics

Me

ch

an

ics

Co

ntr

ol

Systems Engineering

Legend:

Artifact Dependencies: Relations among domain-specific artifacts and system-level artifacts

…

Development Artifacts: Design artifacts Product artifacts

…

…

…

…

…

Business Process …

…

…

Artifacts of varying abstractions and engineering domains Original slide from Peak (GIT), SysML

Presentation 2005

MD VALUE: DESIGN-BUILD-VALIDATE-VERIFY

• Configurations, dependencies,redundancy…

Evaluate architectural design alternatives

• reliability, performance, cost,… Multiple points of

view

• refinement/abstraction, multi-perspective, layering…

Best software engineering practices

MD VALUE: DESIGN-BUILD-VALIDATE-VERIFY

3 – FAST

• Software Factory:

– An organisation for the disciplined development of

software with engineering methods

• The Software Factory leads to innovation in:

The organisational structure Technical Office

The methods and techniques of work Domain Engineering / Application Engineering

The production tools FAST, among others.

Objective

• Differentiate the production means from the actual software production

Strategy

• Creation and development of a specific organisation: Technical Office

Functions

• Domain Engineering

• Create, develop and manage development means

Control

• Reuse

• Productivity

• Performance

What’s the Technical Office in this context?

• Normalisation and standardisation are the principles, the techniques and organisation are the instruments and productivity and quality are the results.

The Paradigm

• Reach high levels (>60%) of code reuse

• Produce software almost independent of its author

• Increase productivity and quality in the production of software

Objectives

• Solve the dilemmas of model based software production

• Models with models, or without models

• Adapt the technology to the factory domain

Strategies

The maturity of the

organisation

The development

culture

The initial workload of

the Technical

Office

Technology stacks

What difficulties can we come across?

New Generation

In 2009 one of the major Core Banking organisations in Spain requests a new generation of Software Factory technologies to Tecnalia. FAST.

What’s FAST ?

FAST is an Eclipse based tool to build Software Factories and not a software factory itself.

Our approach is to build an extremely configurable framework so you can use it to generate industrialised software production platforms

This basically means that you should have or plan to build a family of similar products or developments.

What’s FAST ?

FAST uses models, but models are invisible to the developer.

The code and artifacts generated have no dependencies with the factory that generated them.

You can use our software to build your own domain specific software factories using the tools and guidelines provided with it.

Why use Eclipse at all?

The customer answered this for us by making it a non-functional requirement.

”We are already using tools integrated in eclipse and we want to be able to keep integrating these tools and the new software factory, and anything else we might come up in the future.”

So we needed to build an

And what better way to do it by using eclipse to make it

And since we are using Eclipse, why not make use of every technology available to us?

And many more

Model Driven = EMF?

Safe to think so if you are working in Eclipse

But...

What if the end user does not use EMF models?

What if the end user does not even know what EMF-like models are?

What if the end user does not want to use graphical CASE tools?

There are people who actually do not like them!

What if the end user has its data/models in many different formats like text files and excel sheets?

EMF and models are good but not all the users like EMF or models at all!

Model driven without models?

The fact that the user does not perceive the

existence or interact with anything that they consider models does not prevent that:

What they are using are actually models, they are just not aware of this (nor do they need/should to be).

The internal representation of data and processes within the tool use models.

Technical Office engineers should be able to transform whatever data format to an EMF representation.

FAST capabilities

Generates/Builds things, mostly code but not only. And put traces everywhere!

Allows end users to decide the representation in which that are going to give instructions and parameters to the factory.

Allows easy development of transformations/scripts without relying in any specific transformation/scripting technology.

Allows development of transformations/scripts for non-model savvy people (using plain normal code and algorithms and data)

FAST capabilities

Allows concurrent non-blocking generations

Allows distributed generations and deployments of their factory

Any kind of distributed, not client-server only.

Integrates with distributed repository systems

Allows each company to use the factory as they want

The factory must adapt to the company, not the other way around

Allows data/control/GUI integration with other Eclipse and if possible non-eclipse tools

Which are some of FAST’s self imposed requirements?

Modularity

So we can reuse features developed for this project in other developments.

Standards compliant

OSGi and Java standards are to be followed and never broken.

Reduce third party API level dependencies

The APIs are always presented as Java interfaces and dependencies to libraries outside OSGi and Java SE are avoided.

The experience

It’s possible to use MD for the Technical Office, not exposing models to the developers

Domain Engineering assisted by models and transformations

Application Engineering working on native coding

Providers developed for different languages and sources

In the Core banking particular case:

COBOL developers working 10% to 20% faster than before

60% to 80% of sustainable reuse

Integrated with several Eclipse based tools (e.g. lifecycle)

Thank you very much.

Jason Mansell Productivity technologies lead

ICT/European Software Institute

Tecnalia jason.mansell@tecnalia.com

Joseba Laka Market Director

ICT/European Software Institute Tecnalia Joseba.laka@tecnalia.com

4 – Annex

What is this about OSLC?

• Open Services for Lifecycle Collaboration (OSLC) – OSLC is a new approach to solving the problem of

integrating lifecycle tools • Standardize the way that software lifecycle tools can share

data with one another:

– Requirements

– Defects

– Test cases

– Plans

– Code

– …

– See: http://open-services.net/about/

– Members: http://open-services.net/participate/

Is OSLC just another middleware?

• No, OSLC is not middleware – It does not really define kind of API, instead it reuses already

existing standards.

• Communication is done via HTTP1.1/REST

• Basic semantics are inherited from RDF/DCMI specifications

• OSLC defines extended semantics to allow semantic data transfer between tools.

• OSLC sometimes looks more like a convention rather than a full fledged specification. – Proposes how data should be accessible

– How to create catalogs of the exposed resources

– How to navigate these catalogs

• For an application to be OSLC compliant it does not need to adhere to any middleware or architecture.

What are OSLC domain specifications?

• Definitions of how tools working for a particular domain should share their resources.

• Some finalized OSLC domains: – Change Management – Quality management – Requirements Management – Architecture Management – Software Configuration Management – …

OSLC Core Specification How to define OSLC resources. How to offer services.

How to inform clients of resource shapes. How to offer delegate UIs. How to offer query capabilities. How to offer resource creation. What authentication is allowed. How specification versioning works. How to represent OSLC defined resources.

OSLC Domain Specification Defines OSLC resources. Offers services.

May offer resource shapes. May offer delegate UIs. May offer query capabilities. May offer resource creation. Provides examples of representations.

What is the relationship between IBM Jazz and OSLC?

• Jazz based/powered tools implement OSLC specifications

• Jazz is a middleware – Which implements OSLC services – Facilitates exposing other tools resources as OSLC

• Jazz Integration Architecture (JIA) extends OSLC further – JIA defines Jazz Foundation Services

• Jazz team Server provides an implementation of those.

• Both are IBM initiatives. – Jazz is more IBM centric

• Latest versions of IBM tools moving towards Jazz -> OSLC.

– OSLC is more community centric.

• More information at: http://www.slideshare.net/snoopdave/innovate-2010oslcjazz

Why would someone like to integrate an IDE with FAST?

• Whatever you can do with the tools that are already known you want to keep doing it with the same tools. – Diminishes resistance to change.

• If the GUI is consistent between both tools the learning curve diminishes a lot. – The HMI between the tools uses the same channels and conventions.

• If both tools are integrated into a single IDE the end user only sees one tool. – Keeps thing simpler

• Reduces end user annoyance and frustration. – No need to switch between tools

• If you are generating code, you would like to see it right away in your IDE. – Same for any other thing you are generating and is managed by the

IDE.

• Why not? – Integration is generally a good thing if done correctly.

How can I integrate an Eclipse IDE with FAST?

• Several ways to integrate: – Install FAST modules together with RAD in the same Eclipse

environment • IDE integration • GUI unification • Can create plug-ins based on OSGi services or Java APIs to add

data and control integration to the applications.

– Use OSLC • Example: RAD integrates with IBM Rational Team Concert which is

built upon Jazz – Enables to create RAD plug-ins which can use provided FAST OSLC

services and vice versa.

– No need to be installed in the same Eclipse (or even the same machine)

• Control integration • Data integration • IDE and GUI integration still an option but not achieved freely

– Do both or something else • FAST is extremely flexible and modular

What integration approach is better between an Eclipse IDE and FAST?

• Deploy in the same Eclipse – The IDE

• RAD for example

– FAST’s Generation Client Services • Host APIs for generation management

– FAST’s Generation Client GUI • Eclipse based

– Desired FAST’s Request Grammar Editors • Might be Eclipse Editors • Might be editors based on the installed IDE • Might be external tools

– FAST’s Provisioning services • Resources access and cache

– FAST’s Networking services • Seamless network interaction with at least one communication supported (OSLC)

• Deploy in the same or other Eclipse’s within the organization’s network – FAST’s Generation Server Services

• Generator implementation

– FAST’s Provisioning services – FAST’s Networking services – FAST’s Persistence services

• Automated distributed storage abstraction services (for generation results and requests)

• The IDE and FAST client do not communicate but share the same environment

• FAST clients can be networked via OSLC to other remote FAST servers

Can FAST be used as replacement or complement of cheat sheets for design pattern enforcement?

• Yes! For both questions.

• Replacement – If all the user information for the pattern can be obtained at

the beginning of the guidance

• FAST can be used to create a batch generation

• Complement – If the information cannot be compiled at the beginning of the

process

• Cheat sheet are still a better approach – The automation of each cheat sheet can be implemented via a FAST

generation.

• One of the advantages of software factories is: – Standardization (relative to the organization) of all the

generated code.

• For architectural patterns the benefit is even greater.

What does FAST offer in regards of team collaboration?

• Depends on the definition of Team Collaboration – User management

• Authentication (via JAAS and OSGi User Admin) • Authorization (via OSGi Conditional Permission Admin)

– Process management • FAST offers APIs and services that allow it to be automated by process management and automation

tools. • Deep integration with Mylyn is planned but not critical at the moment.

– OSGi User Admin integration with Mylyn

– Brisges to FAST’s editors to Mylyn.

– Traceability • FAST automatically creates traces of the generations, with complete information about the

inputs/outputs and requesters. • Exposes traces as OSLC links. See: http://open-

services.net/bin/view/Main/OslcCoreSpecAppendixLinks

– Versioning • FAST Provisioning features and modules are being designed to offer seamless management of

versioned resources. – It relies on third party backends

– The integration via Eclipse’s Team Provision is granted.

• FAST is tool designed to allow the implementation of a wide selection of services that a software factory based development process would need. – The tool does not impose any methodology of its own.

• Although it comes with off the shelf support to implement Tecnalia’s approach

What does FAST offer in regards of process enforcement?

• FAST is designed to help automate development processes. – Mainly code generation

– Documentation

– Configuration generation

• It does this by allowing each organization to code their own rules, processes and languages – In their selected coding languages

– In their selected representation models

• Finally it provides a distributed batch generation environment. – All the data is gathered before a generation from one more heterogeneous

sources

– The generation is requested, then processed, the output stored and indexed and the requester notified.

• This allows organizations to ensure the quality of their products by taking developers through semi-automated processes – Generate as much as possible

• If the generators are well built the results will be homogeneous, predictable and easily maintainable.

– Only code where allowed • Furthermore, the special code each program might need can be asked in advance of

the generation and allow it to place it where appropriate.

What integration can be achieved with process management tools/features?

• Defining Users and Roles – FAST uses the OSGi UserAdmin Retrieve/Set the users/roles

• Manage Users (create/update/query) • Manage Roles (create/update/query)

– Each project area in Process Management tools can usually define the set of roles that are available within that project. • Fast provides APIs to automatically synchronize the OSGi

repository with User and Role information Providers. – Event and notification based

• Automation – FAST provides several control integration interfaces to enable

the automation of its usage. • REST services. • OSGi services. • Open Java API. • OSGi Events. • …

What integration can be achieved with team awareness tools/features?

• Tracing each element to the user that

– Generated it.

– Contributed to it.

– Created a GenBot.

– Created a work order.

– …

• Traceability is handled internally by FAST

– Traceability is stored within the log of each generation

– OSLC external link representation is available

• All traces can be represented via links

• These traces can eventually be stored in other tools.

What integration can be achieved with source control tools/features?

• FAST Provisioning Services – Handle versioned resources – Manage local caches for remote resources – Notification management – Automatic synchronization of resources – Querying via URLs – …

• But it need to work against a backend – Eclipse Team Provider APIs ease backend

implementation/integration – Not restricted to it. – Open enough to adapt to any kind of backed and

version control model.

What integration can be achieved with administration tools/features?

• FAST provides administration features: – User/Role management – Authorization for Roles – Network management – Remote FAST configuration. – …

• Most administrative tools in the market provide web frontends for these things – User management and LDAP integration

• FAST authorization rule APIs use OSGi Conditional Permission for their enforcement – Provides Adaptors and APIs to integrate rule sources

and providers.

What is the benefit of integrating FAST with Software Development Process control tools?

• FAST provides services and tools to IMPLEMENT Software Factory related processes – FAST does not define those processes nor does it

manage them

• Software Development Process control tools – Define software processes – Allow automation of those processes

• Integration allows: – Defining and implementing software factory based

development processes. – Intagrating automated generation features to your

existing development processes.

What are the benefits of integrating FAST with testing automation tools?

• Depends on what is tested and how it is tested – There might be little benefit

• or there might be a lot.

• FAST creates software product lines / factories – Product lines create similar products

• One test for everything (little benefit) – Product line usage ensures product compliance to the test

definitions » Ensures relevancy of test across all products.

• Customized tests per customized product (lots of benefit) – Benefit comes from product lining the tests

» For each generated product the tests for that specific product should be generated.

– Need to create TWO product lines » One for the products » One for the tests

What does FAST’s Functional Architecture look like?

Is FAST Service oriented?

It follows one so technically yes, but architecturally not really

OSGi services and compliance is used. But direct calls to the methods of the services are avoided as

much as possible between service implementations.

Event/Command/Message oriented!

OSGi EventAdmin service is used as the main control mechanism.

Events and their respective handlers define the commands/functionality exposed and required by each module

Application is reactive and context aware Easily extensible thanks to the whiteboard pattern employed

by the EventAdmin service

Why the difference between client and server?

Peer 2 Peer (P2P) is the reason.

Do NOT confuse with with Point 2 Point.

In a P2P network/architecture each node can act

as:

a client/consumer of services/functionality

a server/provider of services/functionality

Both simultaneously.

None and act as simple relay

In order to fully decouple the design of the client

from the server we use a messaging infrastructure.

The interfaces of the messages need to be know by both the client and the server

But the client does not know that the server even exists and vice versa

If it is event based why use services at all?

OSGi standards compatibility

The fact that we recommend and only communicate via messages does not mean that there might not be an scenario in the future where the services need to be accessed directly

Division of responsibility

The classes and modules implementing OSGi services focus on providing the functionality of the domain.

Data

Calculus

The EventHandlers are the responsible of calling the appropriate services and their operations to perform a certain command

Security, communications and other horizontal issues are handled at event/message level or above.

Integration with third party tools

There are more tools providing services for control than tools relying on event or messages.

How does FAST know what providers are available?

FAST AUTOMATED FACTORY OSGi service

This is service that we have implemented taking what we liked from the technologies defined in the previous slide.

It basically allows a user of the AutomatedFactory service to request the creation of an instance of an object just knowing the interfaces it needs to implement and the parameters for its construction/initialization.

Also allows querying for how many

implementations are available for a certain interface and notifications.

How does FAST handle any/all data types?

FAST should be able to handle all Java Types

Should because the only way to prove it doesn't is to find a data type it cannot handle which has not happened yet.

Internally it makes heavy use of reflection so basically, as long as reflection is available for a type and the object can be created into memory there

should be no problem to pass it as a parameter to the transformations/bots and use it.

Runtime expression languages are the system that FAST

uses to transform a representation of an object into an

actual Java Object

Thus FAST can handle virtually any data type which has a

Java implementation to handle it.

What are runtime expression languages?

Basically they are plain text or strings which

represent an expression or an object.

They need to be evaluated and upon evaluation they return a Java Object

OCL

XPATH

FAST String Substitution Language (VAR)

FAST Java Constructor Language

Any other....

These languages need to be evaluated in

runtime and for that to happen there must

be plugins providing a FAST

EvaluationContext.

What is a FAST Evaluation Context?

It is the interface that classes need to implement in order to register a new language provider.

Registration takes place automatically using the Automated Factory.

This interface is not simple, and the number of methods that need an actual implementation instead of an empty one will vary depending solely on the nature of the language.

How do you use the expression languages?

Via FAST Expression service

With an ID of the language you are trying to interpret it will allow you to get an Evaluation context for it.

Once you have the evaluation context you can populate it with variables and other contextual information (if relevant).

Then you just need to call the method evaluate with the string you want to evaluate, and optionally the return type you expect and a placeholder for errors for feedback.

If you can specify in you DSLs, GenRequestProviders, or anywhere a method for extracting the ID of the expression language and the expression content as a string the you can use this system.

Which means that if a new expression language gets supported, it will be usable automatically by any user of the FAST Expression service.