generative software engineering - ejcp'2013

107
Generative Software Engineering Part 1 Prof. Dr. Bernhard Rumpe Software Engineering RWTH Aachen University http://www.se-rwth.de/ Farbe!

Upload: others

Post on 11-Feb-2022

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Generative Software Engineering - EJCP'2013

Generative

Software

Engineering

Part 1

Prof. Dr. Bernhard RumpeSoftware EngineeringRWTH Aachen University

http://www.se-rwth.de/

Farbe!

Page 2: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 2

Our Working Groups and Topics

Automotive / Robotics

− Autonomous driving

− Functional architecture

− Variability & product lines

− Requirements engineering

− Simulation

− Robotics

Energy

− Modeling of facilities and

buildings

− Formal planning of functions

− Data management

− Automated analyses

− Quality assurance

− Monitoring

Cloud Services

− Service platforms

− Migration into the cloud

− Evolution of services

− Internet of Services

− Internet of Things

− Tool development

− Tool-Framework MontiCore

− UML, SysML, Architecture DL

− Domain-specific languages (DSL)

− Generation, synthesis

− Testing, Analysis, verification

− Software architecture, evolution

− Agile methods

Model-based Software Development

Page 3: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 3

Language & tooling workbench MontiCore

� Definition of modular language fragments

� Interfaces between models/language fragments

• Name spaces, typing (~ Java, UML)

• „kinds“ + signatures

� Assistance for analysis

� Assistance for transformations

� Pretty printing, editors (graphical + textual)

� Composition of languages:

• independent language development

• composition of languages and tools

• Language extension

• Language inheritance (allows replacement)

� Quick definition of domain specific languages (DSLs)

• by reusing existing languages

• variability in syntax, context conditions, generation, semantics

Page 4: Generative Software Engineering - EJCP'2013

Generative

Software

Engineering

1. Introduction

1.2. Languages for Modeling

Prof. Dr. Bernhard RumpeSoftware EngineeringRWTH Aachen University

http://www.se-rwth.de/

Page 5: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 5

Unified Modeling Language UML

� UML is a second generation notation forobject oriented modeling

� Organized by the OMG (Object Management Group)

(Source: Wikipedia)

Page 6: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 6

Domain Specific Languages (DSL)

� Domain-Specific Languages allow us to abstractly model application domains like

• cars, business workflows, telecommunication, production systems, construction of railway tracks, ...

� Example: Railway track control which uses

• rails, sensors, switches, signals

• A framework provides a software component for every hardware element.

• The configuration is defined by a graphical DSL

SensorSensor

Page 7: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 7

Graphical vs. Textual Representation

� Different representations:

• Graphical vs. Textual

� Which one is more compact?

� Better to assess? To Overview?

� Easier to read/write?

A C

-

® D

B

Æ-

¬E

C = A;

D = !A && B;

E = !A && !B;

© Lehrstuhl für Software Engineering, RWTH Aachen

Page 8: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 8

Definition: Domain-Specific Languages

"A DSL is a language designed to be useful for a limited set of tasks, ...." [JB06]

"A small, usually declarative language expressive over the distinguishing characteristics of a set of programs in a particular problem domain." [DK98]

� A DSL is a language

• which focuses more on the application domain than the technical solution,

• not necessarily executable (if it is executable, it is often not Turing complete),

• and allows describing / solving problems from an application domain in an efficient way.

� DSML = DSL for Modeling (of Software)

Page 9: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 9

Goals of this Lecture

1. Understand what modeling in SE means

2. Understand how to use generative tools

3. Understand how a generator works

4. Understand how a generator is adapted to specific needs

5. Understand how to write a generator for a DSL

Page 10: Generative Software Engineering - EJCP'2013

Generative

Software

Engineering

1. Introduction

1.3. Methodological considerations

Prof. Dr. Bernhard RumpeSoftware EngineeringRWTH Aachen University

http://www.se-rwth.de/

Page 11: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 11

DSL-driven Development

� Domain Specific Modeling Languages (DSML)as a central notation in the development process

DSML models

static analysis

rapid prototyping

code generationautomated tests

refactoring/

transformation

documentation

• DSMLs serve as central notation for development of software

• a DSML can be programming, test, or modeling language

Page 12: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 12

Steps of Code Generation

� AST = abstract syntax tree of model

ModelAST

ModelAST

Text in DSL A

Text-to-Model-Transformation

Model-to-Model-Transformations

Model-to-TextTransformation

...

conforms to conforms to

...

e.g., optimizations orreduction to simpler form

DSL ZDSL A

Text in DSL Z

Parse,context check

pretty print,or templates

transform

Tool

Page 13: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 13

Roles & dependencies in a Generative Project

manually

written code

runtime system

Environment: hardware, GUI, frameworks

generated code

+ included parts

Predefined

componentsPredefined

componentsPredefined

components

API

AP

I

tool provider

application programmer

applicationmodeler

componentprovider

Parame-

terized

generator

model

generatorcustomizer(“toolsmith”)

Generator script/template

Map: concept code

© Lehrstuhl für Software Engineering, RWTH Aachen

Page 14: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 14

Generative Software Engineering

� Generative software engineering (GSE) is a

• Method that uses generators to efficiently generate software systems or parts of software systems from models written in UML or a DSL in order to increase quality and decrease development time.

� If DSLs are used, domain experts can model, understand, validate, and optimize the software system directly.

� UML models or DSLs are used to model certain aspects of a software system in an intuitive and concise manner.

� Of-the-shelf or hand-made generators process the models to generate production and test code.

Page 15: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 15

References (1)

[CE00] K. Czarnecki, U. Eisenecker: Generative Programming: Methods, Tools, and Applications. Addison-Wesley 2000

[DK98] A. v. Deursen, P. Klint: Little Languages: Little Maintenance? J. of Software Maintenance: Research and Practice, 1998

[GK03] J. Gray, G. Karsai: An Examination of DSLs for Concisely Representing Model Traversals and Transformations Proceedings of HICSS, 2003, 325

[GSCK04] J. Greenfield et al. : Software Factories, Wiley 2004

[HGK+08] J. Hartmann; H. Grönniger; H. Krahn; S. Kriebel; L. Rothhardt, B. Rumpe: Modelling Automotive Function Netswith Views for Features, Variants, and Modes.Proceedings of 4th European Congress ERTS 2008

[JB06] Frederic Jouault & Jean Bezivin KM3: a DSL for Metamodel Specification Proceedings of 8th IFIP International Conference on Formal Methods for Open Object-Based Distributed Systems(LNCS 4037), 2006, 171-185

[KMB96] Richard B. Kieburtz; Laura McKinney; Jeffrey M. Bell; James Hook; Alex Kotov; Jeffrey Lewis; Dino P. Oliva; Tim Sheard; Ira Smith & Lisa Walton: A software engineering experiment in softwarecomponent generation. ICSE '96: Proceedings of the 18th international conference on Software engineering, IEEE Computer Society, 1996, 542-552

Page 16: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 16

References (2)

[Rum11] B. Rumpe: Modellierung mit UML: Sprache, Konzepte

und Methodik. 2te Auflage, Springer 2011.

[Rum12] B. Rumpe: Agile Modellierung mit UML: Codegenerierung, Testfälle,

Refactoring. 2te Auflage, Springer 2012.

[RVWL03] G. Rosu; R. Venkatesan; J. Whittle & L. Leustean

Certifying optimality of state estimation programs

Proc. of CAV'03 (LNCS 2725), 2003, 301-314

[SV05] T. Stahl, Markus Völter : Modellgetriebene

Softwareentwicklung: Techniken, Engineering,

Mangement. dpunkt 2005.

© Lehrstuhl für Software Engineering, RWTH Aachen

Page 17: Generative Software Engineering - EJCP'2013

Generative Software

Engineering

2. An initial Example

2.1. UMLs Class Diagrams

Prof. Dr. Bernhard RumpeSoftware EngineeringRWTH Aachen University

http://www.se-rwth.de/

Farbe!

Page 18: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 18

Steps in Developing a Generator

1. Select a modeling language (e.g. Class Diagrams)

• … or design your own DSL

2. Understand the target language (e.g. Java)

• and the frameworks to rely upon (EJB’s, …)

3. Hand-code a concrete model to understand the mapping

• Identify where each concept maps to

• What can/should be optimized?

• Is the result stable? Reliable? Robust? Quick?

• Does the “generated” code scale up to large projects?

4. How to integrate hand written code?

• Please do not allow modifications of generated code

• Define and publish explicit APIs instead

Page 19: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 19

Example Class Diagram from UML

� A small campus management system:

Module

int ident

int credits

Curriculum

String name

Lecture

Grade

float value

int credits

Academic Student

int matrNo

*

*

*

responsible

*

*

1

1

* 1

«abstract»

Person

String name

String email

*

*

CD

Page 20: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 20

Example Class Diagram from UML

� A small campus management system:

Module

int ident

int credits

Curriculum

String name

Lecture

Grade

float value

int credits

Academic Student

int matrNo

*

*

*

responsible

*

*

1

1

* 1

«abstract»

Person

String name

String email

*

*

CD

composition

association with name

cardinality

navigation direction(here: unidirectional) bidirectional navigation

list of attributes:types may be omitted

field for classname

Page 21: Generative Software Engineering - EJCP'2013

Generative Software

Engineering

2. An initial Example

2.2. A manual mapping of Class Diagrams

Prof. Dr. Bernhard RumpeSoftware EngineeringRWTH Aachen University

http://www.se-rwth.de/

Page 22: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 22

Textual versus Graphical Notation of CD

CD

*

*

*

responsible

*

*

1

1

* 1

Module

int ident

int credits

Curriculum

String name

Lecture

Grade

float value

int credits

Academic Student

int matrNo

«abstract»

Person

String name

String email

*

*

� The class diagram is written in textual form

classdiagram Campus {

abstract class Person {

String name;

String email;

}

class Student extends Person {

int matrNo;

}

class Grade {

float value;

int credits;

}

// some parts omitted...

association [*] Grade <-> Student;

}

CD textual

Page 23: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 23

Goal of our Generation Process

� What are our Goals?

� Classes with attributes

� Associations � Attributes

� Get/set-Methods

� Factory

� Manager collects instances

� Framework specific:

• Storage Linearization

• Stubs (if remote) Interface vs. Realization

• (Gui, Testinterface, …)

discussCD

*

*

*

responsible

*

*

1

1

* 1

Module

int ident

int credits

Curriculum

String name

Lecture

Grade

float value

int credits

Academic Student

int matrNo

«abstract»

Person

String name

String email

*

*

Page 24: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 24

Mapping of the Concepts

� To understand the generator, we map the CD concepts manually

� However, we try to be systematic (and not use domain specific knowledge) � the generator will be systematic too

� Main concepts we handle in this example:

• Class

• Attributes

• Associations

• 1-*, 1-1, *-*

• Rolenames

• Navigation

• Inheritance (multiple)?

• Composition

• (there are some more)

• Additional Infrastructure?

CD

*

*

*

responsible

*

*

1

1

* 1

Module

int ident

int credits

Curriculum

String name

Lecture

Grade

float value

int credits

Academic Student

int matrNo

«abstract»

Person

String name

String email

*

*

Page 25: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 25

� Here a version with

• get/set-methods and

• hidden attributes

� Advantage:

• subclasses may redefine functions

Classes and Attributes

public class Curriculum {

private String _name;

public String getName() {

return this._name;

}

public void setName(String name) {

this._name = name;

}

}

Curriculum

String name

CD

Java

Page 26: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 26

� This association can easily be handled locating the links on the „*“-side

� There we need to storeonly one link per object

� Note that class Module knows nothing about Grade!

Associations 1--*

Module

int ident

int credits Grade

float value

int credits

*1

// Association *--1 to Module

private Module _module;

public Module getModule() {

return this._module;

}

public void setModule(Module module) {

this._module = module;

}

CD

Java

Page 27: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 27

Associations *--*

exercise

Module

int ident

int credits

Curriculum

String name

*

*

� What to do in this case?

� What do we expect?

� (lets first look at the Curriculum-side only)

� Links can be added, deleted,

• Is link contained?

• Iteration over links

• Get# of Links, empty Link

• Link to be added only once

� Set of links: SET-Functionality

� Navigability in both directions

• Redundant storage

• Consistency at all times

CD

Page 28: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 28

Set (from http://docs.oracle.com/javase/6/docs/api/java/util/Set.html)

// from java.util

interface Set extends Collection {

boolean add(Object o)

void clear()

boolean contains(Object o)

boolean isEmpty()

Iterator iterator()

boolean remove(Object o)

int size()

// ... more

}

// ... We might also use the generic version Set<E>

Java

Page 29: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 29

Set (from http://docs.oracle.com/javase/6/docs/api/java/util/Set.html)

� How to provide this Set-functionality to Curriculum?

// from java.util

interface Set extends Collection {

boolean add(Object o)

void clear()

boolean contains(Object o)

boolean isEmpty()

Iterator iterator()

boolean remove(Object o)

int size()

// ... more

}

Approach:

• integrate to class signatureProblem:

• several associations

Solution:• domain specific renaming

Java

Module

int ident

int credits

Curriculum

String name

*

*

CD

Page 30: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 30

Signature of a *-Association (adapted from Set)

� Type-specific adaptations of Set-functionality in class Curriculum

private Set _modules = new HashSet();

boolean addModule(Object Module o)

void clearModules()

boolean containsModule(Object Module o)

boolean isEmptyModules()

Iterator iteratorModules()

boolean removeModule(Object Module o)

int sizeModules()

// ... more

Association stored as set of links

Association name extends signature(necessary when several associations occur)Observe the added “ s“ and caption adaptations!

Module

int ident

int credits

Curriculum

String name

*

*

CDJava

Page 31: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 31

Implementation of a *-Association

// from class Curriculum

boolean addModule(Module o) {

return _modules.add(o);

}

void clearModules() {

_modules.clear();

}

boolean containsModule(Module o) {

return _modules.contains(o);

}

// ... more All methods are implemented through Delegation

Module

int ident

int credits

Curriculum

String name

*

*

CDJava

Page 32: Generative Software Engineering - EJCP'2013

Generative Software

Engineering

2. An initial Example

2.3. Additional Infrastructure

Prof. Dr. Bernhard RumpeSoftware EngineeringRWTH Aachen University

http://www.se-rwth.de/

Page 33: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 33

Additional Infrastructure: Factory

� Factory is used to create objects (instead of new…)

� Adds flexibility, e.g., to replace class by subclass

• for testing, for monitoring, for technological considerations

� Example:public class StudentFactory {

public static Student create(String name, String email, int matNo) {

if (factory == null) {

factory = new StudentFactory();

}

return factory.doCreate(name, email, matNo);

}

protected static StudentFactory factory = null;

protected Student doCreate(String name, String email, int matNo) {

return new Student(name, email, matNo);

}

}

publicly accessible factory method

factory

calls normal constructor

Java

Page 34: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 34

Additional Infrastructure: Manager

� Manager stores and retrieves existing objects

• collaborates with the factory

� Example:public class StudentManager extends StudentFactory {

// Manager also provides Factory functionality

public static void initialize() {

if(manager == null) {

manager = new StudentManager();

factory = manager;

}

}

protected static StudentManager manager = null;

protected Student doCreate(String name, String email, int matNo) {

Student o = new Student(name, email, matNo);

students.add(o);

return o;

}

// Manage set of students

private Set students = new HashSet();

establishes itself as factory

overrides factory method

manages Set of all Students

Java

Page 35: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 35

Additional Infrastructure

� More to come:

• Managers,

• Database table creation, SQL queries

• Mapping the objects to the database / retrieval

• Serialization

• SOA-access and manipulation functions to the data

• Web-interface to access, create, and manipulate the data

Page 36: Generative Software Engineering - EJCP'2013

Generative Software

Engineering

3. Template-Based Generation

3.1. Introduction

Prof. Dr. Bernhard RumpeSoftware EngineeringRWTH Aachen University

http://www.se-rwth.de/

Farbe!

Page 37: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 37

Types of Generators

� Template-based generation

• template refers to elements of the abstract syntax

• simple pieces of code embedded in templates

• target language with embedded holes

• holes are pieces of tool code to be executed and inserted when

generating

• control structures in the template language allow e.g. to handle lists or call sub-templates

• readable; relatively easy to change

• beneficial if generated code structure is similar to model

� API-based

• API allows to build internal representation of the target

• last step: pretty print to text

Page 38: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 38

� Consider the following example of a concrete automaton and its AST:

Abstract Syntax:

Tool-internal Representation of the Model

AST-OD

:ASTAutomaton

name = "MyAut"

:ASTState

final = true

name = "B"

:ASTState

inital = true

name = "A"

:ASTTransition

source = "A"

target = "B"

Abstract Syntax Treein form of an Object Diagram

Transition source and target defined through names

A

B

AutomatonMyAut

Page 39: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 39

� Deriving the AST-CD from one (or several examples):

Using Class Diagrams to represent the

AST in the Tool

AST-CD

ASTState

boolean initial

boolean final

String name

ASTAutomaton

String name

*

ASTTransition

String source

String target

*

AST-OD:ASTAutomaton

name = "MyAut"

:ASTState

final = true

name = "B"

:ASTTransition

source = "A"

target = "B"

Page 40: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 40

� For tools the model-details need to be accessible

� For this we define an abstract representation of the model in form of a CD

� The resulting object structure of a concrete model is called AST (Abstract Syntax Tree)

� BTW: A grammar can define an AST as well: Automaton ::= Name State* Transition*

State ::= initial:Boolean final:Boolean Name

Transition ::= source:Name target:Name

Representation of Models in Tools

AST-CD

ASTState

boolean initial

boolean final

Name name

ASTAutomaton

Name name

*

ASTTransition

String source

String target

*

Page 41: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 41

Summary

� The tool dealing with a modeling language L needs an internal representation

� This internal representation can be visualized by CD‘s for the structure and OD‘s for concrete models.

� We operate on this internal structure.

� CD’s describing languages are also called “Metamodels”.

� When generating Code from Class Diagrams we need their metamodel

• and thus model CD’s using itself …

Page 42: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 42

**

The AST of the CD language - complete

ASTModifier

boolean public

boolean private

boolean protected

boolean abstract

AST-CD...

ASTCDClass

String name

ASTCDAttribute

String name

ASTModifier modifier

ASTCDConstructor

String name

ASTModifier modifier

ASTCDMethod

String name

ASTModifier modifier

* * *

ASTCDAssociation

String name

boolean aggregation

boolean composition

boolean bidirectional

ASTCDParameter

String name

ASTQualifiedName

List<String> parts

right

Referenceleft

Reference

ASTCDDefinition

String name

**

«interface»

ASTType

«interface»

ASTReturnType

Page 43: Generative Software Engineering - EJCP'2013

Generative Software Engineering

3. Template-Based Generation

3.2. Template Engine: Freemarker

Prof. Dr. Bernhard RumpeSoftware EngineeringRWTH Aachen University

http://www.se-rwth.de/

Farbe!

Page 44: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 44

conforms to

Examle: Generation of Java Code from UML/P

Classdiagrams with Freemarker

� Automatic transformation of specification into executable Java code in two steps

1. create representation of specification in main memory (AST)

2. generate code in target language (Java) from AST

Factory.ftl

Freemarkertemplatelanguage

conforms to

CDDSL

Java

MyCD.cd CDParser

MyCDAST

FreemarkerEngine

conforms to

MyClassFactory.java

Page 45: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 45

Freemarker – Overview

� Code generation defined by combination of AST and template

� In our case: Template is called with AST as data-model

� Example:

� Interpretation of the result,e.g., as HTML:

ast:ASTAutomaton

name = "MyAut"

The name of the

automaton is

<b>${ast.getName()}</b>.

The name of the

automaton is

<b>MyAut</b>.

AST Template Result

+ =

The name of the automaton is

MyAut.

fixed content Freemarker directive:Pieces of Java are processed by template engine

Page 46: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 46

Freemarker Example: Generating a Factory

FM Factory.ftl Generated/Java

package ${ast.printPackage()};

<#assign cname = ${ast.printName()}>

public class ${cname}Factory {

protected static

${cname}Factory f = null;

protected ${cname}Factory() {}

public static ${cname} create() {

if (f == null) {

package a.b;

public class PersonFactory {

protected static

PersonFactory f = null;

protected PersonFactory() {}

public static Person create() {

if (f == null) {

� ${ast.printPackage()}:

• Java-like Expressions resulting in String

� <#assign cname = ${ast.printName()}>:

• Definition and assignment to local variable

Page 47: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 47

Freemarker – Types

� Freemarker provides some basic types:

� Scalars

• String, e.g. "Hello World" or 'Hello World'

• Number, e.g. 15 or -3.7 (no distinction between int, double, …)

• Boolean, reserved words: true and false

• Date, e.g. "10/25/1995"?date("MM/dd/yyyy")

� Containers

• Hash: Container with pairs of key:valuee.g. {"name":"mouse", "price":50}

• Sequence: Container with single valuese.g. ["foo", "bar", 12.3]

date-format

untyped: values can beof different scalars

Page 48: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 48

Freemarker – Build-ins for Types

� Each type has several additional built-ins:Format: exp?functionname

� Examples:

� String:

• "abcde"?substring(1, 3) equates to "bc"

• "abcde"?cap_first equates to "Abcde"

• "abcde"?contains("de") equates to true

� Number:

• 2.5?round equates to 3

• 2.5?floor equates to 2

� Sequence:

• ["c", "a", "b"]?first equates to "c"

• ["c", "a", "b"]?seq_index_of("a") equates to 1

• ["c", "a", "b"]?sort equates to ["a", "b", "c"]

Page 49: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 49

Freemarker – Variables and Conditions

� Variable declaration: <#assign name=value>

� Variable access: ${name}

� Condition: <#if condition> …

<#elseif condition2> …

<#else> … </#if>

� Example:

<#assign i=4>

<#assign j=8>

<#if (i==j) >

${i} and ${j} are equal

<#elseif (i<j) >

${i} is smaller than ${j}

<#else>

${i} is bigger than ${j}

</#if>

4 smaller than 8

Template Result

=

Page 50: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 50

Freemarker – Loops

� Loop: <#list sequence as item> … </#list>

� For loops Freemarker provides two special “operations/variables”:

• _index: index of the current item

• _has_next: true if not at the end of the sequence

� Example:

:ASTState

initial = true

name = "A"

ast:ASTAutomaton

name = "MyAut"

:ASTState

final= true

name = "B"

States of ${ast.getName()}:

<#list ast.getStates() as s>

${s_index + 1}) ${s.getName()}

<#if s.isInitial()>

is initial

</#if>

</#list>

States of Aut:

1) A is initial

2) B

AST-OD Template Result

+ =

Page 51: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 51

Freemarker – Switch and Include

� Switch: <#switch value><#case refValue1> … <#break>

<#default> …

</#switch#>

� Include: <#include path>

� Example:

Hello World <html>

<body>

Hello World

</body>

</html>

templates/Content.ftl Result

=+

<html>

<body>

<#include "templates/Content.ftl">

</body>

</html>

templates/Body.ftl

Page 52: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 52

Two Objects providing Information

� MontiCore provides two objects with various methods:

� ast: current AST-node (varies)

• to navigate through the AST (e.g. to print children etc.).

� op: predefined Operations from the Template-Operator

• provides a multitude of additional methods for handling templates

op:TemplateOperator

+ Object getValue(String name)

+ setValue(String name, Object value)

+ callTemplates(…)

+ String includeTemplates(…)

+ setDefaultFileExtension(String extension)

+ callCalculator(String calculatorName)

OD

storage for additional data(global to the model)

call and embedding of templates

...

Page 53: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 53

TemplateCalculator for Complex Computation

� Should be used if more complex computation is needed and external data has to be accessed

public class MyTeCalculator extends TemplateCalculator<ASTCDType>{

@Override

public boolean calc(ASTCDType ast, TemplateOperator op) {

// ... any complex computation

op.setValue("varName", arbitraryObject);

return true;

}

}

Man. Java@Tool

extends abstract TemplateCalculator class and is parametrized with an ASTNode class

Must override calc method. Return value can beused e.g. in if-block to determine if content of template should be included

<#if op.callCalculator("MyTeCalculator")>

// ...do something, include templates etc.

</#if>

Templatequalified name of calculator

Page 54: Generative Software Engineering - EJCP'2013

Generative

Software

Engineering

Part 2

Prof. Dr. Bernhard RumpeSoftware EngineeringRWTH Aachen University

http://www.se-rwth.de/

Farbe!

Page 55: Generative Software Engineering - EJCP'2013

Generative Software

Engineering

6. MontiCore, Frontend

6.1. Grammar and AST

Prof. Dr. Bernhard RumpeSoftware EngineeringRWTH Aachen University

http://www.se-rwth.de/

Farbe!

Page 56: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 56

MontiCore - Overview

� Compact grammar-based definition of DSLs

� Framework-based solution

� Easy integration of additional components

� Assists generative development

� Assists:

• parsing, AST, transformation

• symbol-tables (name + type analysis)

• pretty-printing, editor-generation

• …

� Independency

• available as command line tool, Eclipse-plugin or online-service

www.monticore.de

Page 57: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 57

Running Example: Finite Automata

automaton PingPong {

state NoGame <<initial>>;

state Ping;

state Pong;

NoGame - startGame > Ping;

Ping - stopGame > NoGame;

Pong - stopGame > NoGame;

Ping - returnBall > Pong;

Pong - returnBall > Ping;

}

analyzeAutomaton

NoGame

Ping

Pong

startGamestopGame

returnBall

returnBall

stopGame

Page 58: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 58

Example: Finite AutomataDefinition as MontiCore Grammar (1/2)

grammar Automaton {

Automaton = "automaton" Name

"{" ( State | Transition )* "}" ;

State = "stt e" Name:Name

( "initial"] ">>" | "< "final"] )*

";" ;

Transition =

From "-" Input:Name ">" To

� Abstract syntax

• nonterminals (e.g. Transition)

• semantically relevant constants (e.g. “initial“)

• identifiers (e.g. Name, From)

MG

Page 59: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 59

Example: Finite AutomataDefinition as MontiCore Grammar (2/2)

grammar Automaton {

Automaton = "automaton" Name

"{" ( State | Transition )* "}" ;

State = "state" Name

( ["initial"] ">>" | "<<" ["final"] )*

";" ;

Transition =

from:Name "-" input:Name ">" to:Name ";" ; }

� Abstract syntax extended with type information:

• constants (e.g. “initial“) are ensured to have an identifier

• identifiers (e.g. Name) get a type (e.g. Name)

MG

© Lehrstuhl für Software Engineering, RWTH Aachen

Page 60: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 60

Automatic Derivation of Abstract Syntax (1/3)

State

Automaton

Transitiona class for eachnonterminal

AST-CD

MGgrammar Automaton {

Automaton = "automaton" Name

"{" ( State | Transition )* "}" ;

State = "state" Name

( ["initial"] ">>" | "<<" ["final"] )*

";" ;

Transition =

from:Name "-" input:Name ">" to:Name ";" ; }

Page 61: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 61

Automatic Derivation of Abstract Syntax (2/3)

State

initial: boolean

final: boolean

name: String

Automaton

name: String

Transition

from: String

input: String

to: String

* *

identifiermaps to

String

a constantis mapped to a boolean

AST-CD

MGgrammar Automaton {

Automaton = "automaton" Name

"{" ( State | Transition )* "}" ;

State = "state" Name

( ["initial"] ">>" | "<<" ["final"] )*

";" ;

Transition =

from:Name "-" input:Name ">" to:Name ";" ; }

Page 62: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 62

Automatic Derivation of Abstract Syntax (3/3)

State

initial: boolean

final: boolean

name: String

Automaton

name: String

Transition

from: String

input: String

to: String

* *

nonterminal in the ruleis handled as composition

AST-CD

MGgrammar Automaton {

Automaton = "automaton" Name

"{" ( State | Transition )* "}" ;

State = "state" Name

( ["initial"] ">>" | "<<" ["final"] )*

";" ;

Transition =

from:Name "-" input:Name ">" to:Name ";" ; }

Page 63: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 63

Adding Concrete Syntax (1/2)

grammar Automaton {

Automaton = "automaton" Name

"{" ( State | Transition )* "}" ;

State = "state" Name

("<<" ["initial"] ">>" | "<<" ["final"] ">>" )*

";" ;

Transition =

from:Name "-" input:Name ">" to:Name ";" ; }

� Concrete syntax:

• syntactic sugar through

• unnamed constants (e.g. “automaton“)

• separators (e.g. “;“ , “<<“)

� Result: Definition of abstract and concrete syntax for the DSL

MG

Page 64: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 64

Adding Concrete Syntax (2/2)

grammar Automaton {

Automaton = "automaton" Name

"{" ( State | Transition )* "}" ;

State = "state" Name

("<<" ["initial"] ">>" | "<<" ["final"] ">>" )*

";" ;

Transition =

from:Name "-" input:Name ">" to:Name ";" ; }

� AST is a tree

• nonterminals are nodes / objects

• terminals are leaves / attributes

� Problem:

• no connection between related elements

• e.g. transition and its source/destination state

� Solution: additional associations

MG

Page 65: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 65

Efficient Navigation in ASTs

� Two approaches to implement naming concepts:

� Symbol table describes a mapping of name to defining position, additional information like type etc.

• each name space has its own symbol table

• symbol tables can be nested

:Automaton

Name = "PingPong"

:Transition

from = "NoGame"

input = "startGame"

to = "InPlay"

*

AST-OD:Symboltable

ASTNode find(String)

:State

initial = False

final = False

name = "InPlay"

:Transition

from = "Ping"

input = "returnBall"

to = "Pong"

:State

initial = True

final = False

name = "NoGame"

name

„NoGame“ „InPlay“

Page 66: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 66

Efficient Navigation in ASTs

� Second approach:

� Direct links between objects of the AST

• links from usage of a name to its definition

� Easier to define, but less flexible, e.g. in adding information or nesting name spaces.

� Result: Graph with an embedded spanning tree

:Automaton

Name = "PingPong"

:Transition

from = "NoGame"

input = "startGame"

to = "InPlay"

*

AST-OD

:State

initial = False

final = False

name = "InPlay"

:Transition

from = "Ping"

input = "returnBall"

to = "Pong"

:State

initial = True

final = False

name = "NoGame"

fromStatetoState

……

Page 67: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 67

Linking Objects

� association extends the structure of the AST classes

� concept simplereference defines how links of this structure

are to be filled.

� simplereference can be used if we have a flat, simple name

space.

association Ingoing

Transition.ToState * <-> 1 State.IngoingTransitions;

association Outgoing

Transition.FromState * <-> 1 State.OutgoingTransitions;

concept simplereference {

Ingoing: Transition.to -> State.name;

Outgoing: Transition.from -> State.name;

}

MG

© Lehrstuhl für Software Engineering, RWTH Aachen

Page 68: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 68

Finite Automaton: Complete Definition

grammar Automaton {

Automaton =

"automaton" Name

"{" ( State | Transition )* "}";

State =

"state" Name

("<<" ["initial"] ">>"|"<<" ["final"] ">>")* ";";

Transition =

from:Name "-" input:Name ">" to:Name ";";

concept simplereference {

Ingoing: Transition.to -> State.name;

Outgoing: Transition.from -> State.name;

}

}

MG

Page 69: Generative Software Engineering - EJCP'2013

Generative Software

Engineering

6. MontiCore, Frontend

6.4. The Grammar Reference: Productions

Prof. Dr. Bernhard RumpeSoftware EngineeringRWTH Aachen University

http://www.se-rwth.de/

Page 70: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 70

Terminals

MG

� A terminal is normally ignored in the AST

� When semantically relevant e.g. terminals in alternatives

� terminal becomes an attribute

AST-CD

A = "Hello"

(who: "World" | who:"Tom")

"!";

A

who: String

Page 71: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 71

Optional Terminals

MG

� Optional single values become a Boolean flag

� keyword is either present or absent

AST-CD

A = ["initial"]?;A

initial: boolean

Page 72: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 72

Enumerations 1

MG

� Multiple values become an integer

� constants describe the possible values

� several concrete representations can map to the same constant

� if name is omitted, the name is extracted from the constant (e.g. PUBLIC from “public”).

AST-CD

A =

vis:[ PUBLIC:"+" | "public" |

PRIVATE:"-" | "private"];

Constants

DEFAULT = 0;

PUBLIC =1;

PRIVATE = 2

A

vis: int

multi values are represented by ints

Page 73: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 73

Enumerations 2

MG

� Explicit definition of an enumeration nonterminal

� it only has alternatives with terminals, no additional internal structure

� several concrete representations can map to the same constant

AST-CD

enum VISIBILITY =

PUBLIC:"+" | "public" |

PRIVATE:"-" | "private" ;

A = vis:VISIBILITY;

«enum»

VISIBILITY

PUBLIC

PRIVATE

A

vis: VISIBILITYenumerations

Page 74: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 74

Composition of the AST

MG

� In productions the nonterminals are mapped to compositions

� single occurring nonterminals: simple attribute

� multiple occurring nonterminal: List

� If the AST shall differ from the standard AST, it can be explicitly defined through the “ast” statement.

AST-CD

S AS = x:A y:A*;

A = ...;

1x

*y

S AS = y:A*;

A = ...;

ast S =

y:A* min=n max=m;

n..m

y

nonterminals result into compositions

Page 75: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 75

Subclassing in the AST

MG

� Nonterminals can be defined to explicitly extend a given nonterminal

� S extends A

� Effect:

� AST: S-objects are allowed where A-objects are required

� parsing: Each production for A is also valid for S

� Means: when A is expected, S can be parsed.

AST-CD

S AS extends A = "bar" y:A*;

A = "foo" ;

*

y

S inherits all productions of form„A = …“ (in addition to those defined explicitly for S)

Page 76: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 76

Sequences of Nonterminals in the AST

MG

� Nonterminals with the same name are grouped together

� useful e.g. in the case below

� All elements with the same name also need the same type, e.g. “y:A y:B*” is not allowed

� Automatic analysis determines whether an attribute has a single (simple attribute) or multiple values (list)

AST-CD

S = x:A "::=" y:A ("," y:A )*; S Ax*

y

1

Page 77: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 77

Automatic Naming

� Automatic naming of unnamed nonterminals:

• “bla:Bla” can be abbreviated by “Bla”

� Effect:

• omits unnecessary names

• makes the grammar more compact

Automaton =

"automaton" Name "{"

( State | Transition )*

"}" ;

equivalent

MG

MG

Automaton =

"automaton" name:Name "{"

( state:State | transition:Transition )*

"}" ;

Page 78: Generative Software Engineering - EJCP'2013

Generative Software

Engineering

6. MontiCore, Frontend

6.5. The Grammar Reference: Syntax Tree

Prof. Dr. Bernhard RumpeSoftware EngineeringRWTH Aachen University

http://www.se-rwth.de/

Page 79: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 79

Additional Attributes

grammar Automaton {

Automaton = "automaton" Name

"{" ( State | Transition )* "}" ;

State = "state" Name

("<<" ["initial"] ">>" | "<<" ["final"] ">>" )* ";" ;

Transition = from:Name "-" input:Name ">" to:Name ";" ;

ast State =

reachableStates:State*;

}

MG

State

initial: boolean

final: boolean

name: String

reachableStates:List<State>

Automaton

name: String

Transition

from: String

input: String

to: String

* *

additional attributein the abstract syntax(not part of the concrete syntax)

AST-CD

Page 80: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 80

Additional Methods

� The ast-statement allows

� to control the AST e.g. when it shall differ from what can be derived from the concrete syntax

� to add methods

grammar Automaton {

State = ... ;

ast State =

reachableStates:State*;

method public boolean isReachable(State s) {

return reachableStates.contains(s);

}

}

MG

State

reachableStates:List<State>

public boolean isReachable(State s)

Automaton

*

AST-CD

...

...

additional method(including implementation)

Page 81: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 81

Inheritance:Abstract Syntax

«interface»I A

A

S A

A = ...;

interface I;

A implements I = ...;

A = ...;

interface I = A;

S = ...;

A extends S = ...;

or

«interface»

I

«interface»

J

interface J extends I;

interface I;

MG AST-CD

Page 82: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 82

MontiCore AST

� The content of text files is parsed and transferred to an object structure (the AST)

� The AST-classes are automatically derived from the grammar

� The object structure

• is easier to process than the text file

• represents the essential content of the text file(without line breaks and spaces)

• is strongly typed (access through named attributes)

• allows the traversal by the Visitor pattern

Page 83: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 83

MontiCore AST: Common Base Classes

«interface»

ASTList

boolean _existent

Comment

SourcePosition

«interface»

ASTNode

void traverse(Visitor)

ASTNode deepClone()

ASTCNode

ASTCList

pre

post *

*

start

end 1

1

parent

Visitor

ConcreteVisitor

1

*

AST-CD

Page 84: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 84

Lessons learnt

� MontiCore

• compact grammar-based definition of DSLs

• concrete and abstract syntax are defined in an integrated format

• abstract syntax contains inheritance and associations

• language inheritance and embedding permits compositional modular language development

• grammar format is extensible by concepts(e.g. for generating editors or establishing links)

Page 85: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 85

For Comfort: Editor in Eclipse

� Normally developing editors in Eclipse is time-consuming!

� MontiCore allows to declaratively specify the necessary information

concept editorattributes {

keywords: automaton, state, initial, final, counter;

foldable: Automaton;

segment: Automaton ("pict/a.gif")

show: "Automaton " Name; // ... }

MG

Page 86: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 86

References

[GHJV96] E. Gamma, R. Helm, R. E. Johnson, J. Vlissides: Design Patterns: Elements of ReuseableObject-Oriented Software. Addison-Wesley, 1996

[GKR+08] H. Grönniger, H. Krahn, B. Rumpe, M. Schindler, S. Völkel MontiCore: A Framework for the Development of Textual Domain Specific Languages In: 30th International Conference on Software Engineering (ICSE). Companion Volume.Leipzig, Germany, 2008

[KRV08] H. Krahn, B. Rumpe, S. VölkelMontiCore: Modular Development of Textual Domain Specific Languages In: Proceedings of the 46th International Conference Objects, Models, Components, Patterns (TOOLS-Europe).(LNBIP 11) Springer-Verlag 2008

[KRV07a] H. Krahn, B. Rumpe, S. VölkelEfficient Editor Generation for Compositional DSLs in EclipseIn: Proceedings of the 7th OOPSLA Workshop on Domain-Specific Modeling (DSM' 07), Montreal, Quebec, Canada, Technical Report TR-38, Jyväskylä University, Finland

[KRV07] H. Krahn, B. Rumpe, S. VölkelIntegrated Definition of Abstract and Concrete Syntax for Textual Languages In: Proceedings of the ACM/IEEE 10th International Conference on Model Driven Engineering Languages and Systems (MODELS 2007), Nashville, TN, USA, October 2007G. Engels et al.(Eds.): LNCS 4735, pp. 286-300, Springer-Verlag Berlin-Heidelberg 2007

[GKR+06] H. Grönniger, H. Krahn, B. Rumpe, M. Schindler, S. VölkelMontiCore 1.0 - Ein Framework zur Erstellung und Verarbeitung domänenspezifischer SprachenInformatik-Bericht 2006-04, Technische Universität Braunschweig, Carl-Friedrich-Gauss-Fakultätfür Mathematik und Informatik, 2006

Page 87: Generative Software Engineering - EJCP'2013

Generative Software

Engineering

12. Compositional Language Development

12.1. Background and Motivation

Prof. Dr. Bernhard RumpeSoftware EngineeringRWTH Aachen University

http://www.se-rwth.de/

Farbe!

Page 88: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 88

Observations on languages (1):

� DSLs have a specific purpose

� and describe a specific aspect/view of the system

• UML:

• CD for structure,

• Statechart for behavior,

• deployment diagram for distribution

• typical: we need several languages to describe the whole system

� combination of languages often project-specific

• e.g. Statecharts with Java or C++ as action language

� some languages can be combined with others

• OCL-constraints with CD, OD, Java, …

Page 89: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 89

Observations (2):

� Combination of languages can happen

• within one file: CD with embedded OCL

• in separated files: OD and SD

• or both: CD + embedded Java and pure Java-files

� Sometimes we need a specialized DSL

• but DSL may be similar to an existing language

• e.g. CDs with special stereotypes; Java-Statements + Extras

� combinations are not necessarily fixed

• e.g. CD + Statecharts, where

• states become enumeration values of objects,

• states become classes (state pattern), or

• a statechart governs the communication between classes

Page 90: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 90

Typical Approach

� Simple example:

� We have a grammar + generator + editor for

• CD + Java Block-Statement for method bodies + OCL for pre-/postconditions

• a symbol table/CoCo-check for this combination

• a language-driven editor/IDE for this combination

� Next project: Use C++ instead of Java

• copy everything and change/reimplement => bad

� Solution: development self-contained, independent language collections and compose (instead of copy-paste)

Page 91: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 91

Compositional Language Development

� Instead of strongly coupled languages:

� Compose the language you need from existing sub-languages

� Central idea: languages as components which can be

• developed independently

• combined flexibly and late with minimum effort

Page 92: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 92

Language Composition

� Use cases:

• language embedding

• OCL embedded in CDs

• Action language (Jave) embedded in Statecharts

• language inheritance (“refinement”) and extension

• add special stereotypes to ClassDiags

• extend Java 1.3 -> 1.4 (introduction of assert-statement)

• language aggregation: defining a language family

• UML (= CD + Statecharts + …)

• here the files are kept separate, but packages contain models of

different forms, that 2fit” together

Page 93: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 93

Dimensions of Language Composition

� Dimensions:

• concrete syntax• How do the integrated artifacts look like? Parser?

• abstract syntax• How does combined internal representation look like?

• symbol tables• How to create combined symbol tables?

• context conditions• New CoCos emerging from composition? Inter-language CoCos?

• language-driven editors• With language-dependent functionalities.

• semantics (=meaning)• (Not dealt with here.)

• code generation & synthesis• (Not dealt with here.)

� Note: each dimension should be developed independently & easily composed without changing/having the sources.

� Important: each mechanism must be supported in all dimensions!

Page 94: Generative Software Engineering - EJCP'2013

Generative Software

Engineering

12. Compositional Language Development

12.2. Concrete and Abstract Syntax

Prof. Dr. Bernhard RumpeSoftware EngineeringRWTH Aachen University

http://www.se-rwth.de/

Page 95: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 95

Extending the Automaton with Actions

grammar AutomatonWithAction extends Automaton {

TransitionWithAction extends Transition =

from:Name "-" input:Name

"/" Action ">" to:Name ";" ;

}

AutomatonWithAction inherits all rulesfrom super-grammar

TransitionWithAction can substitute Transition (subtyping)

� Code for new AST, parser, etc. is generated without modifying the Automaton-parser

• AST is reused

• parser is reused

• visitor, associations, etc. are reused

MG

© Lehrstuhl für Software Engineering, RWTH Aachen

Page 96: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 96

� Extended nonterminal leads to new subclass

� Extension of a language does not affect the old AST:

� many functions can simply be reused

AutomatonWithAction: Abstract Syntax

State

initial: boolean

final: boolean

name: String

Automaton

name: String

Transition

from: String

input: String

to: String

IngoingTransitions

ToState

OutgoingTransitions

FromState *

*

* *

TransitionWithAction

action: Action

AST-CD

Page 97: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 97

Automaton with External Actions

grammar AutomatonWithAction2 extends Automaton {

external Action;

TransitionWithAction extends Transition =

from:Name "-" input:Name

"/" Action ">" to:Name ";" ;

}

� Nonterminal “Action” is declared external

� It can be used within the grammar, but is not defined there

� Code for new AST, parser, etc. is generated

• allowing independent compilation

� AST is reused

� Parser is reused

� Visitor, associations, etc. are reused

Action is placeholder for an embedded nonterminal

MG

Page 98: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 98

Composition of Languages

� Language definition in form of fragments

• with explicit parameterization (holes) of form “external nonterminal”

� Combination of fragments to a language

• by explicit binding the open “holes” in a configuration

� Tooling

• combination of language tools to a coherent tool:

• including the parser and the AST

Page 99: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 99

Embedding = Language & Tool Composition

grammar AutomatonWithAction extends Automaton {

external Action;

TransitionWithAction extends Transition =

from:Name "-" input:Name

"/" Action ">" to:Name ";" ;

}

nonterminal Action isexternal(extension point)

grammar AutomatonWithAction {

…}

lexer/parser

lexer/parsergeneration

lexer/parser

AutomatonWithAction.Action =Java.Statement

text file, e.g.

...a – y / x=5 -> b...

grammar Java { Statement =

Assignment | … ;}

generation

configured by Java-API orconfiguration script

MG

Page 100: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 100

Late Embedding of Languages

� Goal:

• independent generation of parsers for single languages

� Allows us to come up with a component infrastructure for tools

grammar 1

X = “out“ “{“ A “}“;

external A;

grammar 2

S = “test“;lexer/parser

lexer/parser

generation

generation

combined lexer/parser

configuration:grammar1.A ->grammar2.S

text file:

out { test }

processing

© Lehrstuhl für Software Engineering, RWTH Aachen

Early, independent generationBy third party

Combination without re-generation/re-compilatione.g. in the project

MG 2

MG 1

text

Page 101: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 101

Abstract Syntax:

Automatons with Java as Action Language

State

initial: boolean

final: boolean

name: String

Automaton

name: String

Transition

from: String

input: String

to: String

IngoingTransitions

ToState

OutgoingTransitions

FromState *

*

* *

TransitionWithAction<< interface >>

Statement

...

Java

Automaton

AutomatonWithAction

Action

AST-CD

Page 102: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 102

Language Aggregation

� Concrete and abstract syntax:

• nothing to do, because files remain independent

� For each language MontiCore generates AST-classes

• independent of any other language in use

� Instances of the AST-classes are produced by the parser

� MontiCore:

• one tool may comprise several parsers, lexers etc. for artifacts in different languages

• to handle them in parallel

Page 103: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 103

Lessons learnt

� Compositionality means

• understanding the system by understanding the parts separatelyand understanding the composition operator

� Compositional software means

• reuse instead of development from scratch or copy-paste-modify

• separate development of components

• integration of existing components without recompilation

� Compositionality requires

• standardization

• adaptation

• interfaces

• self-contained components and glue

Page 104: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 104

Lessons learnt

� MontiCore supports three kinds of compositionality

• embedding

• inheritance/extension

• language aggregation

� These kinds are consistently supported for

• concrete syntax

• abstract syntax

• symbol tables

• context conditions

• editors

Page 105: Generative Software Engineering - EJCP'2013

Generative Software

Engineering

15. Wrapup

Prof. Dr. Bernhard RumpeSoftware EngineeringRWTH Aachen University

http://www.se-rwth.de/

Page 106: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 106

Bootstrapping for DSL Development

� Bootstrapping

• process of creating a DSL with a prior version of the same DSL

� Pro

• "eat your own dog food"

• usually increases quality as DSL is used

• using higher abstraction of DSL for the development

� Con

• can be confusing

• requires versioning and storage of older versions

• compiler cannot be compiled without prior version of itself

• DSL must be expressive enough to describe its own tooling

Page 107: Generative Software Engineering - EJCP'2013

Prof. Dr. B. Rumpe

Lehrstuhl für

Software Engineering

RWTH Aachen

Seite 107

We wish you successful generative development projects …

manually

written code

runtime system

Environment: hardware, GUI, frameworks

generated code

+ included parts

Predefined

componentsPredefined

componentsPredefined

components

API

AP

I

tool provider

application programmer

applicationmodeler

componentprovider

Parame-

terized

generator

model

generatorcustomizer("toolsmith")

Generator script/template

Map: concept code

Rep.