domain specific languages · 2014-06-21 · 23.10.2010 3 general purpose domain specific...

23.10.2010

1

Domain Specific LanguagesExamples, Tools, Practices

Markus Voelterindependent/itemis

Introduction

Domain Specific LanguagesExamples, Tools, Practices

A short

of programmingHistory

AbstractionAbstraction is a conceptual process by

which higher, more conceptual concepts are derived from the usage and

classification of literal (i.e. "real" or "concrete") concepts.

Wikipedia

23.10.2010

2

Abstractions may be formed by reducing the information content of a concept or an

observable phenomenon, typically to retain only information which is relevant for a

particular purpose.

Wikipedia

[..] to understand and solve problems and communicate their solutions with the computer in

some particular computer language. It allows programmers to separate concepts from

implementation, so that they do not depend on software or hardware.

Wikipedia

[..] to understand and solve problems and communicate their solutions with the computer in

some particular computer language. It allows programmers to separate concepts from

implementation, so that they do not depend on software or hardware.

Wikipedia

One Language to Rule Them All?

What is a good Language? What is a good Language?

… for a given purpose?

23.10.2010

3

general purpose

domain specific

effective++

specialized, limited

used by experts

together with otherspecialized tools

tailor made

A DSL is a focussed, processablelanguage for describing a specific concern when building a system in a specific domain. The abstractionsand notations used are natural/suitable for the stakeholders who specify that particular concern.

execute?

23.10.2010

4

map

DSL Program

GPL Program

(aka Model)

map

automated!

map

GenerationTransformationCompilation

Interpretation

Analysing DomainsDefining Languages

Activities

Building EditorsTransforming ModelsBuilding Generators

Building Frameworks

Adapting/Selecting

23.10.2010

5

Analysing DomainsDefining Languages

Activities

Building EditorsTransforming ModelsBuilding Generators

Building Frameworks

Adapting/Selecting

… and using all of that to build apps

internal

external

vs.

customization

configuration

vs.

graphical

textual

vs.

23.10.2010

6

Examples

Domain Specific LanguagesExamples, Tools, Practices

Example 1:

Embedded Protocol Handler

Co

mp

on

ent

Spec

ific

atio

n

Me

ssag

e F

orm

at D

efin

itio

n

tests

refines

Test

ing

23.10.2010

7

Example 2:

Pension Fund Specification

Text

ual

Do

cum

enta

tio

n

Insu

ran

ce M

ath

em

atic

s

Cal

cula

tio

nR

ule

s an

dTe

sts

Example 4:

Alarm System Menus

23.10.2010

8

Men

u S

tru

ctu

re

Soft

war

e C

om

po

nen

ts

Example 5:

OSGi-based System

Co

mp

on

en

tSp

ecif

icat

ion

Test

Cas

e M

od

elin

g

23.10.2010

9

De

plo

ymn

et

Example 8:

Miscellaneous

SOLU

TIO

N

A DSL Hearing Aid Configuration

SOLU

TIO

N

A DSL forRefridgeratorConfiguration

BP

EL D

esi

gne

r

23.10.2010

10

Blo

ck D

iagr

ams

PLC

Pro

gram

min

g

Stat

e C

har

ts

Concepts & Terms

Domain Specific LanguagesExamples, Tools, Practices

Metamodel (Abstract Syntax)

23.10.2010

11

Metamodel (Abstract Syntax)

Concrete Syntax (Notation)

Metamodel (Abstract Syntax)

Concrete Syntax (Notation)

Static Semantics(Constraints)

Metamodel (Abstract Syntax)

Concrete Syntax (Notation)

Static Semantics(Constraints)

Dynamic Semantics

Metamodel (Abstract Syntax)

Concrete Syntax (Notation)

Static Semantics(Constraints)

Dynamic SemanticsModel vs. Program

Benefits

Domain Specific LanguagesExamples, Tools, Practices

23.10.2010

12

Automationfaster, deterministic

Increased Qualitywell defined structures allthrough the system

Meaningful Validationmore semantics in the model

CaptureDomain Knowledge

formalized into languages and models

Suitable Notationstextual, graphical, tabular

Technology Independencegenerate „technology glue code“

23.10.2010

13

Abstraction w/oRuntime Overhead

generator „optimizes away“

Implementation StrategyCapture

in the generators

Tooling

Domain Specific LanguagesExamples, Tools, Practices

Tooling! Tooling!Editor

23.10.2010

14

Tooling!Editor, Debugger

Tooling!Editor, Debugger, Testing

Tooling!Editor, Debugger, Testing, Groupware

Tooling!Editor, Debugger, Testing, Groupware, Scalable

Tooling!Editor, Debugger, Testing, Groupware, Scalable, „All in Eclipse“

23.10.2010

15

Tools Tooling

Language Definition Tools

Editor FrameworksTransformation Languages

Code Generation Tools

abstract syntax, concrete syntax, constraints

Tooling: Eclipse Xtext

Domain Specific LanguagesExamples, Tools, Practices

Xtext

Building Textual Editors

Xtext

Building Textual Editors

23.10.2010

16

Constraints

expressedin Java-or-with OCLand dialects

Xtext Typesystem F/W

TypingRulesFor Expr’s

M2M

Model-to-Model Transformations

INRIA’s ATLQVTXtend

M2T

Model-to-Text Transformations

JET: Java Emitter Templates

Xpand: oAW’stemplate engine

GMF

Graphical Box/Line editors based on EMF

EMF

Ecore meta meta model+

EditingTransactionsValidation

QueryDistribution/Persistence

23.10.2010

17

DEMO

Building DSLs with Eclipse Xtext

Programming vs. Modeling

Domain Specific LanguagesExamples, Tools, Practices

programming?

modelling?

23.10.2010

18

modelling?

programming?vs.

the same?

different?

Modeling Tools

Programming Tools!=

Different Worlds

Modeling Tool

Modeling Tool!=

Different Worlds Different WorldsMix Models and Programs

23.10.2010

19

Different WorldsAST Navigation & Query

Mix Models and Programs

Different WorldsAST Navigation & Query

Mix Models and Programs

Code Constraints

Modeling Programming

Modeling Programming

… (Mostly) Textual Notations

… Concrete Syntax Storage

… (Fancy) ASCII Editors

… Read-Only Visualizations

Modeling

… (Mostly) GraphicalNotations

… Abstract Syntax Storage

… Projecting Editors

… Different editableviews for model

Programming

… (Mostly) Textual Notations

… Concrete Syntax Storage

… (Fancy) ASCII Editors

… Read-Only Visualizations

23.10.2010

20

It is time for …

… a Different Perspective

We don‘t want to

we want tomodel,

program!

We don‘t want to

we want tomodel,

program!… at different levels of abstaction

… from different viewpoints

… integrated!

23.10.2010

21

We don‘t want to

we want tomodel,

program!… with different degrees of

… with suitable notations

… with suitable expressiveness

domain-specificity

We don‘t want to

we want tomodel,

program!

precise and tool processableAnd always:

Modular Languages

Domain Specific LanguagesExamples, Tools, Practices

L

ab

c

d

e

f

gh

i

j

k

m

n

o

with many first class concepts!

Big Language?

L

Small Language?

and poweful conceptswith a few, orthogonal

23.10.2010

22

my L

a b c

d e f

g h i

j k l

Modular Language

composable conceptswith many optional,

Like frameworksand libraries,

Modular Language

Like frameworksand libraries,

but with syntaxand IDE support

Modular Language Viewpoints

Viewpointssuitable

abstractions

notationsfor each

and

Viewpoints

Integrated

references

transitions

symbolic

seamlessand

via

23.10.2010

23

ViewpointsGeneral Purpose

predefinedlibrary

configure

ViewpointsDomain Specific

custompurpose-built

create/include

Viewpoints

CustomNotations

businessexpert integration

real

Domain SpecificViewpoints

C

LEGO Robot Control

General Purpose

Domain Specific

Viewpoints

C

LEGO Robot Control

Components

State Machines

Sensor Access

General Purpose

Domain Specific

23.10.2010

24

Language Integration by…

ReferencingLanguage Integration by…

Cascading

Language Integration by…

ExtensionLanguage Integration by…

Embedding

Language Integration by…

Contributing

23.10.2010

25

Not a new idea…

Growing A Language(Guy L Steele)

Growing A Language(Guy L Steele)

stoneage-OOPSLA talk,get it on Google Video!

LanguageWorkbench

(Martin Fowler)

LanguageWorkbench

(Martin Fowler)

Freely

define

integratethem

languages and

23.10.2010

26

LanguageWorkbench

(Martin Fowler)

usepersistent

abstractrepresentation

LanguageWorkbench

(Martin Fowler)

language ::=

schemaeditors

generators+

+

LanguageWorkbench

(Martin Fowler)

editingprojectional

LanguageWorkbench

(Martin Fowler)

persist

incomplete

contradictoryinformation

or

LanguageWorkbench

(Martin Fowler)

powerful

editingtesting

refactoringdebugging

language definition

IDE definitionimplies

+groupware

LanguageWorkbench

(Martin Fowler)

support for„classical“

programming„classical“

modelingand

+

23.10.2010

27

Not a new idea… Not a new idea…

But

a practicalreality :-)

Projectional Editing

Domain Specific LanguagesExamples, Tools, Practices

Parser-basedtext… to tree… to text

Projectionaltree… to text-lookalike (editor)… to other trees … **+… to text

Tree

23.10.2010

28

Programming as Modeling

… (Mostly) GraphicalNotations

… Abstract Syntax Storage

… Projecting Editors

… Different editableviews for model

Programming as Modeling

… (Mostly) Graphical Any kind ofNotations

… Abstract Syntax Storage

… Projecting Editors

… Different editableviews for model

Language

There‘s no parsing.

Unique Language Element Identity.

Unlimted language composition.

CompositionTextual

Graphical

Semi-Graphical

treated the same

can be mixed

like ASCII }box & line

mathematical

Flexible

Notations

Automatic

IDE Extensiontool support is inherent

for languages build withprojectional tools

language definition

IDE definitionimplies

MultipleNotations

… for the same concepts

e.g. in different contextsor for different tasks

23.10.2010

29

PartialProjections

… different views… for different roles/people… only a particular variant

Storage!= Schema

… independent of language schema!variability annotations

… store arbitraty meta datachange logconflicting information

… „aspects“, overlay

ProgramsLive

think: spreadsheet

a change to one part of programcan lead to (dependent) changesin other parts

Tree Editing

… try to make it feel like text

… takes some getting used tobut: for more flexible notations a more general editing paradigm is needed

… is different from editing text

InfrastructureIntegration

… diff/merge must be in tool

… existing text tools don‘t work

… storage is not text

ProprietaryTools

… no interop

… no standards

23.10.2010

30

Tooling: MPS

Domain Specific LanguagesExamples, Tools, Practices

IntelliJ IDEAResharper

also do…

Q3 2009released in

1.5currently

2.0April 2011

Apache 2.0licensed under

Open Source

&me

&

&

23.10.2010

31

Build new standalone DSLs

Build new standalone DSLsBuild DSLs that reuse parts

of other languages

Build new standalone DSLsBuild DSLs that reuse parts

of other languages

(MPS comes with BaseLanguage)

extend base language

Java++

Build new standalone DSLsBuild DSLs that reuse parts

of other languages

(MPS comes with BaseLanguage)

extend base languagebuild DSLs that reuse parts

of BaseLanguage

Java++

23.10.2010

32

Demo Scenario

Domain Specific LanguagesExamples, Tools, Practices

C is goodfastflexiblewide-spreadmulti-platform

C is badlimited abstractionsnot domain-specific„dangerous“preprocessornot extensible

Modelingcompensates for C‘s

problems

Modelingcompensates for C‘s

problems, specifically

good abstractionsdomain-specificextensible

Modelingcompensates for C‘s

problems, but:not extensibletool/tool integrationtool/code integrationprocess integration

23.10.2010

33

Two Classes

(yes, this is a slight simplification)

Plan OrientedTop Down

Big SystemsBig Companies

Modeling

GrownBottom Up

Small/Medium SystemsSmaller Companies

C Code

Two Classes

… and good luck withmixing the two!

mbeddr.comModular Embedded Language

Incremental Extension

Cof

23.10.2010

34

Incremental Extension

Cof ComponentsTasksState MachinesPhysical UnitsSpecial Data Types

Incremental Extension

Cofsyntacticallyand semanticallyintegrated

Incremental Extension

Cofextensible withdomain-specificconstructs (DSLs)

SOLU

TIO

N

SOLU

TIO

N

23.10.2010

35

SOLU

TIO

N

SOLU

TIO

N

Imported Requirements

SOLU

TIO

N

Program Code with Annotations (green)

SOLU

TIO

N

Selecting from the Requirements

Find Usages of Requirements

SOLU

TIO

N

23.10.2010

36

SOLU

TIO

N

SOLU

TIO

N

DEMO

Extending LanguagesWith MPS

Tool: Spoofax

Domain Specific LanguagesExamples, Tools, Practices

23.10.2010

37

Traditionally:Language ::=

Token* | Rule*;

Traditionally:Language ::=

Token* | Rule*;

Token ::=Character*;

Traditionally:Language ::=

Token* | Rule*;

Token ::=Character*;

Rule ::=Token* | Rule;

Traditionally:Language ::=

Token* | Rule*;

1. Global Tokenization

2. Grammar Recognition

2 Phases:

(aka Scanning)

(aka Parsing)

What if!combine languages

same seq<Char> asdifferent tokens?

Ambiguous!

Not Parsable.

Bad Luck.

23.10.2010

38

Modern:Language ::=

Rule*;

Rule ::=Char* | Rule*;

Grammar Recognition1 Phase:

directly on char*

ScannerlessParsing

No Ambiguity

Always Parsable.

when combining languages

(well, almost. There are some corner cases…)

Custom Syntax

GraphicalTextualSymbolic++

IDESupport

TeamworkDebuggingCustom Editors

CompleteSymbolicIntegration

Goto DefFind RefsRefactoring

UnicodeLimited to

how to handlenon-character symbols

23.10.2010

39

TextGraphics !=

two worlds…

separate editors… per syntax/viewpoint… models can still be ref integrated

Eelco Visser‘s group Eelco Visser‘s group

Eelco Visser‘s group

23.10.2010

40

Recap: What did we do?

Domain Specific LanguagesExamples, Tools, Practices

Abstraction

Make Explicit

Viewpoints

Formalize Notation

23.10.2010

41

IsolateTechnology

LimitFreedom

Automate

A few Best Practices

Domain Specific LanguagesExamples, Tools, Practices

Limit Expressiveness

23.10.2010

42

Notation, Notation, Notation Graphical vs. Textual

DSL Semantics Viewpoints

Partitioning Evolution

23.10.2010

43

The Fallacy ofGeneric Languages Learn from 3GLs

Tooling Matters!Interpretation

Generationvs.

Rich Domain Specific Platform

Integrating Generatedand Manually Written Code

23.10.2010

44

Model-2-ModelTo Simplify Generators

Don‘t forgetTesting

Iterate!Documentationis still necessary

Domain UsersProgramming?

23.10.2010

45

web email skypetwitter

xinglinkedin

www.voelter.devoelter@acm.orgschogglad@markusvoelter

http://www.xing.com/profile/Markus_Voelterhttp://www.linkedin.com/pub/0/377/a31

.coordinates

THE END.

web email skypetwitter

xinglinkedin

www.voelter.devoelter@acm.orgschogglad@markusvoelter

http://www.xing.com/profile/Markus_Voelterhttp://www.linkedin.com/pub/0/377/a31

.coordinates