building dsls with the spoofax language workbench
DESCRIPTION
Slides for my keynote talk at Code Generation 2010TRANSCRIPT
Building DSLs with The Spoofax Language Workbench
Eelco VisserDelft University of Technology
http://eelcovisser.org
Joint work with Lennart Kats, Karl-Trygve Kalleberg, Maartje de Jonge, and many others
The Meta-Languages Wars
Internal DSLs vs External DSLs
Textual vs Graphical
Partial vs Complete Code Generation
Project-Specific vs Multi-Project DSLs
Code Generation vs Interpretation
Dynamically vs Statically Typed
The Meta-Languages Wars
Internal DSLs vs External DSLs
Textual vs Graphical
Partial vs Complete Code Generation
Project-Specific vs Multi-Project DSLs
Code Generation vs Interpretation
Dynamically vs Statically Typed
WebDSLhttp://webdsl.org
separation of concerns & linguistic integration
Eelco Visser. WebDSL: A Case Study in Domain-Specific Language Engineering. GTTSE 2007: 291-373
correct & extend
publication records
authorprofiles
bibliographies
tagging
reputation system
access control rules
user groups
conference calendar
community engineering
etc.
18,000 lines of WebDSL code
138 (generated) tables in mysql
Concerns in web applications
- Data model
- User interface templates
- Actions
- Access control rules
- Data validation
WebDSL
- Linguistic integration
- Cross concern, static consistency checking
WebLib: a model of researchr
Data Model
entity Publication { key :: String (id) title :: String (name,searchable) authors -> List<Author> journal :: String (searchable) volume :: Int issue :: Int year :: Int (searchable) month :: String (searchable) firstpage :: Int lastpage :: Int cites -> Set<Publication> citedBy -> Set<Publication> (inverse=Publication.cites) }
entity Author { key :: String (id) name :: String publications -> Set<Publication> (inverse=Publication.authors) created :: DateTime modified :: DateTime}
User Interface
define page search() { var query : String action show() { var pubs : List<Publication> := searchPublication(query); if(pubs.length > 0) { replace(results, results(query,pubs)); } else { replace(results, noresults()); } } main{ section{ header{"Search"} form{ input(query)[onkeyup:=show()] } placeholder results{} } }}
Full Text Search
entity Publication { title :: String (name,searchable) journal :: String (searchable) year :: Int (searchable) month :: String (searchable) ... }
define page root() { main{ section{ header{"Welcome to WebLib: Your Digital Library"} aboutWeblib() recent() tips() } }}
define aboutWeblib() { par{ "WebLib is a digital library ..." }}
Page & Template Definitions
XML Embeddingdefine no-span main() { <div class="page-wrap"> top() <div class="body"> messages() elements() </div> <div class="clear"></div> </div> footer()}
Page Navigationdefine tips() { section{ header{"What Can You Do Here?"} list{ listitem{ navigate(newauthor()){ "Add an author" } } listitem{ navigate(newpublication()){ "Add a publication" } } listitem{ navigate(search()){ "Search" } } } }}
define page search() { ... }
HQL Embedding
extend entity Person { publications -> List<Publication> := select p from Publication as p, Author as a where (a.person = ~this) and (p = a.publication)}
Forms & Data Bindingdefine page editpublication(pub : Publication) { main{ section{ header{"Edit Publication '" output(pub.title) "'"} form{ group{ label("Key:" ){ input(pub.key) } label("Title:" ){ input(pub.title) } label("Authors:"){ input(pub.authors) } ... } submit action{ pub.modified := now(); return publication(pub); } { "Save" } } } }}
Access Control
Danny M. Groenewegen, Eelco Visser. Declarative Access Control for WebDSL: Combining Language Integration and Separation of Concerns. ICWE 2008: 175-188
Access Control Rules
define showReview(review : Review) { action delete() { ... } section{ ... navigate(editReview(review)){"[Edit]"} submitlink delete() {"[X]"} }}
Access Control Rules
access control rules
principal is User with credentials username, password
predicate isReviewer(review : Review) { review.reviewer == securityContext.principal } rule template showReview(review : Review, pubLink : Bool) { review.public || isReviewer(review) rule action delete() { loggedIn() && isReviewer(review) } } rule page editReview(review : Review) { loggedIn() && isReviewer(review) }
define showReview(review : Review) { action delete() { ... } section{ ... navigate(editReview(review)){"[Edit]"} submitlink delete() {"[X]"} }}
Data Validation
Danny M. Groenewegen, Eelco Visser. Integration of Data Validation and User Interface Concerns in a DSL for Web Applications. SLE 2009: 164-173
Data Validation: Form Input Validation
label("Acronym (without year)"){ input(acronym){
validate(!isEmptyString(acronym), "Provide acronym")
validate(!(/[\ ]/.find(acronym)), "Acronym should not contain spaces")
validate(!(/[0-9][0-9][0-9][0-9]/.find(acronym)), "Acronym should not contain year")
}}
Data Validation: Data Invariants
entity Publication { key :: String (id, index(25),
validate(isUniquePublication(this), "that key is already in use"),
validate(isValidKeyForURL(key), "Key should consist of letters, digits, : and -")) ... }
Data Validation: Assertions
extend entity Person { invitation -> Invitation (inverse=Invitation.invitee) function invite(email : Email, invitation : WikiText) : Invitation {
validate(findUser(email) == null && findInvitationByEmail(email).length == 0, "That email address is already in use.");
... }}
Concerns in web applications
- Data model
- User interface templates
- Actions
- Access control rules
- Data validation
WebDSL
- Linguistic integration
- Cross concern, static consistency checking
IDEs for DSLs
Model
Code
AST
core
Errorsparse check
desugar
generate
Traditional Compiler Architecture
Compiler Ingredients
Syntax definition
★ concrete syntax
★ abstract syntax
Static semantics
★ error checking
★ name resolution
★ type analysis
Model-to-model transformation
★ express constructs in core language
Code generation
★ translate core language models to implementation
IDEs Required
Editor Services for Modern IDEs
Syntactic Editor Services
★ syntax checking
★ bracket matching
★ syntax highlighting
★ code folding
★ outline view
Semantic Editor Services
★ error checking
★ reference resolving
★ hover help
★ code completion
★ refactoring
Editor
Code
AST
core
IntegratedDevelopmentEnvironment
feedback
parse
Architectural Requirements for IDEs
Editor services are based on AST
★ services analyze structure, not text of source
Error recovery
★ continue services in presence of syntactic errors
Incremental processing
★ effort of applying analysis, transformation, generation should be proportional to change in source
Separate compilation (analysis, transformation)
★ keep track of dependencies
Holy Grail of Software Language Definition
Automatically derive efficient, scalable, incremental compiler + usable IDE from high-level, declarative language
definition
The Spoofax Language Workbench
The Spoofax/IMP Language Workbench
Syntax definition: SDF
★ declarative, modular syntax definition
Transformation, analysis, generation: Stratego
★ rewrite rules, strategies, dynamic rules
Editor services
★ domain-specific languages for configuration
Based on IMP framework by IBM Research
Syntax Definition
Syntax Definition
Parse Table Signature Pretty-Print Table
TransformParse Pretty-Print
entity User { name :: String pw :: Secret}def output(u : User) {
@Entityclass User { String _user; public User getUser() { return _user; }syntax definition is basis of language definition
Lexical & Context-Free Syntax
composed from
Structure of Sentences
Context-Free Grammar
grammar production
symbol
DefinitionProperty*
{Stat “;”}*“entity”
regular expressionsort
Lexical Syntax
character class
characters in lexical syntax not separated by layout
regular expression
Syntax of Data Models
RecognizingWell-Formed
Sentences
Lexical Syntax: Identifiers & Literals
follow restriction: symbol may not be followed by character from this class: BlogEntry is one ID
reserved words
Lexical Syntax: Strings
complement
escape
escape escape
Lexical Syntax: Whitespace
LAYOUT? is inserted between symbols in CF productions
normalize
Modular Syntax Definition
Parsing & Abstract Syntax
building structured representation of sentence
Linking Concrete & Abstract Syntax
Terms to Represent Abstract Syntax
Entity(“Blog”, [Property(“name”,SimpleType(“String”)), Property(“author”, SimpleType(“User”))])
Parsing in Spoofax
Signatures
defining the structure of abstract syntax terms
Syntax Definition
Parse Table Signature Pretty-Print Table
TransformParse Pretty-Print
entity User { name :: String pw :: Secret}def output(u : User) {
@Entityclass User { String _user; public User getUser() { return _user; }syntax definition is basis of language definition
Signatures
declare arguments and types of constructorssignature is generated from grammar
Disambiguation
Arithmetic Expressions
Plus(Times(Var(“x”),Int(2)),Var(“z”))note: this style produces good abstract syntax
Associativity
Priority
Parsing after Disambiguation
Brackets
Language Composition
define page index() { list{ for(p : Post in select p from Post as p order by (p.title)) { listitem{ output(p) } } }}
Embedding HQL
ForElem("p", SimpleType("Post"), QueryRule( SelectFrom( Some(Select(None(), [AliasedExpression(Path(["p"]), None())])) , FromClause([FromRangeJoin(FromClass(Path(["Post"]), Some(AsAlias(Alias("p"))), ) , Some(OrderByClause([OrderElement(Paren([Path(["p", "title"])]), None())])) ), [CallElems("listitem", [CallArgs("output", [Var("p")])])])
module expressionsimports Commonimports HQLexports context-free syntax SelectStatement -> Exp
Presentational
Editor Services
Editor Services
Syntax highlighting
★ token coloring
Code folding
★ folding & unfolding code fragments
Outline view
★ table of contents
Syntax properties
★ bracket matching, indentation
Syntax completion
★ match on syntactic triggers & replace with template
module nwl.main
imports nwl-Builders nwl-Colorer nwl-Folding nwl-Outliner nwl-References nwl-Syntax nwl-Completions
language General properties name : nwl id : nwl extends : Root description : "Spoofax/IMP-generated editor for the nwl language" url : http://strategoxt.org extensions : nwl table : include/nwl.tbl start symbols : Start
Editor Services Composition
Syntax Definition
Default Syntax Highlighting
Default Code Folding
DefaultOutline View
Editor
Custom Syntax Highlighting
CustomCode Folding
CustomOutline View+ + +
module nwl-Foldingimports nwl-Folding.generatedfolding Element.CallElems Element.Call Element.ForElem Element.ForAllElem
module nwl-Folding.generatedfolding Default folding Start.Module Definition.Entity Property.Property Definition.TemplateDef Element.CallArgs PageRef.PageRef Element.Action
Code Folding
Outline View
module nwl-Outliner.generatedoutliner Default outliner Start.Module Definition.Entity Property.Property Definition.TemplateDef Element.CallArgs Exp.MethodCall Element.CallElems Element.Call Element.ForElem Element.ForAllElem PageRef.PageRef Element.Action Statement.For Statement.While Statement.If Element.Submit Element.XmlElem
module nwl-Colorer.generated colorer Default, token-based highlighting keyword : 127 0 85 bold identifier : default string : blue number : darkgreen var : 255 0 100 italic operator : 0 0 128 layout : 100 100 0 italic colorer System colors darkred = 128 0 0 red = 255 0 0 darkgreen = 0 128 0 green = 0 255 0 darkblue = 0 0 128 blue = 0 0 255 cyan = 0 255 255 magenta = 255 0 255 yellow = 255 255 0 white = 255 255 255 black = 0 0 0 gray = 128 128 128 grey = gray orange = 255 165 0 pink = 255 105 180 brown = 139 69 19 default = _
Default Syntax Highlighting
Custom Syntax Highlighting
module nwl-Colorerimports nwl-Colorer.generatedcolorer Element : darkgreen
Syntax Properties
module nwl-Syntax.generatedlanguage Syntax properties (static defaults) // Comment constructs: line comment : "//" block comment : "/*" * "*/" // Fences (used for matching, // inserting, indenting brackets): fences : [ ] ( ) { } // Automatic indent hints // (indent after these tokens): indent after : "=" ":" // Regular expression for identifiers: identifier lexical : "[A-Za-z0-9_]+"
Parse Error Recovery
Parse Error Recovery
also for composite languages
BuildersSemantic Editor Services
Builders: Analysis & Transformation Services
module nwl-Buildersbuilders provider: include/nwl.ctree
observer: editor-analyze
builder : "Generate Java code" = generate-java (openeditor) (realtime)
builder : "Show ATerm (selection)" = generate-aterm (openeditor) (realtime) (meta) builder : "Normalize (selection)" = show-normalized (openeditor) (realtime) (meta)
builder : "Normalize Pretty (selection)" = show-normalized-pp (openeditor) (realtime) (meta)
Term Rewriting
Term Rewriting
Term rewrite rules
★ transform term to term
★ pattern matching
★ variable binding
★ substitution
Rewriting strategy
★ algorithm for applying rewrite rules
Term Rewrite Rule
desugar : Property(x, t) -> Property(x, t, [])
label/name
left-hand side pattern
right-hand side patternvariable
Rewrite Strategy
desugar-all = innermost(desugar)
strategy definitiongeneric strategy
strategy instantiation
innermost(s) apply transformation s exhaustively to all sub-terms of subject term in bottom-up order
Constant Folding
y := a + (3 * 5 + 2);
Assign( Var("y"), BinOp(Var("a"), "+", IntLit("17")))
Assign( Var("y"), Plus( Var("a") , Plus(Times(IntLit("3"), IntLit("5")), IntLit("2")) ))
y := (a + 17);
desugar + eval
parse
pretty-print
Constant Folding Rules
strategies
eval-all = innermost(desugar + eval)
rules eval : BinOp(IntLit(x), "+", IntLit(y)) -> IntLit(<addS>(x, y)) eval : BinOp(IntLit(x), "-", IntLit(y)) -> IntLit(<subtS>(x, y)) eval : BinOp(IntLit(x), "*", IntLit(y)) -> IntLit(<mulS>(x, y)) eval : BinOp(IntLit(x), "/", IntLit(y)) -> IntLit(<divS>(x, y))
Conditional Rewrite Rules
eval : BinOp(IntLit(x), "+", IntLit(y)) -> IntLit(<addS>(x, y))
eval : BinOp(IntLit(x), "+", IntLit(y)) -> IntLit(z) where z := <addS>(x, y)condition
match apply transformation
bound in condition
Code Generation by Model Transformation
elem-to-java-servlet : tc@elem|[ x(e)[passign*]{elem*} ]| -> <wrap-input-render-java-code> bstm* |[ String x_temp = ident+"~inputident"+uniqueid; ~*<render-error-messages-with-error-template(|java:expr|[ x_temp ]|, <ErrorTemplateInput>)> bstm*|[ bstm_call* ]| ]| where <is-input-template> tc with inputident := <get-inputnumber> tc ; x_temp := <newname> "temp" ; bstm_call* := <control-flow-tcall-helper(|"render",expr|[ x_temp ]|)> tc
Rewriting with Concrete Object Syntax
Stratego + embedded Java
Consistency Checking
Consistency Checking
Syntax definition
★ what are well-formed sentences?
Static analysis
★ not all ‘well-formedness’ properties are context-free
★ consistency of compositions
★ consistency of expressions wrt declarations
Error reporting
★ indicate errors in editor
★ use sensible error message
Consistency Checking: Ingredients
Editor Interface
★ collecting and displaying errors, warnings
Error checking
★ checking static constraints and reporting errors
Type analysis
★ computing types of expressions
Name resolution
★ disambiguation of names
Reference resolving
★ linking identifiers to declarations
Consistency Checking: Generic Approach
Rename
★ make identifiers unique
Map
★ map identifiers to declarations
Project
★ compute properties of declarations, expressions
Check
★ check constraints
Editor Interface
module nwl-Builders
imports nwl-Builders.generated
builders
provider: include/nwl.ctree
observer: editor-analyze
module static-analysis
imports include/nwlimports entitiesimports utils
rules // static analysis
editor-analyze: (ast, path, fullpath) -> (errors, warnings, notes) with ...
editor/nwl-Builders.esvtrans/static-analysis.str
Editor Interface
editor-analyze: (ast, path, fullpath) -> (errors, warnings, notes) with errors := <collect-all(check, conc)> ast; warnings := <collect-all(constraint-warning, conc)> ast; notes := <collect-all(constraint-note, conc)> ast
Error Checking Rules
check : context -> (target, error) where assumption where require(constraint)
require(s) = not(s)
– Context: identifying points in the code to check– Assumptions: only report an error if certain assumptions hold (validating the context and avoiding spurious errors)– Constraints: checking for constraints at the context– Formulating an error message– Attribution of the error to a particular character range in the source text (usually, only part of the context
Error Checking: Binary Operators
check : e@BinOp(e1, op, e2) -> (e, $[operator [op] not defined for [<pp>t1] and [<pp>t2]]) where t1 := <type-of> e1 where t2 := <type-of> e2 where require(<type-of> e)
Pretty-Printing with String Interpolation
pp : Entity(x, prop*) -> $[entity [x] { [<map(pp)>prop*] }] pp : Property(x,t) -> $[[x] : [<pp>t] ] pp : SimpleType(x) -> x pp : SetType(t) -> $[Set<[<pp> t]>] pp : [] -> $[] pp : [t] -> <pp>t pp : [t1,t2|ts] -> $[[<pp>t1],[<pp>[t2|ts]]]
Type Analysis
type-of : e -> t
compute type of expression
Type Analysis: Literals
type-of : StringLit(x) -> SimpleType("String") type-of : IntLit(x) -> SimpleType("Int")
Type Analysis: Binary Operators
type-of : BinOp(e1, op, e2) -> t where t := <function-type>(op, [<type-of>e1, <type-of>e2])
function-type : ("+", [SimpleType("String"), SimpleType("String")]) -> SimpleType("String") function-type : ("+", [SimpleType("Int"), SimpleType("Int")]) -> SimpleType("Int")
function-type : ("-", [SimpleType("Int"), SimpleType("Int")]) -> SimpleType("Int")
Type Analysis: What is Type of Variable?
define page root(x : Int) { action exptest() { for(y : Int in {1,2,x}) { x := x + y; } } } type-of :
Var(x) -> t where t := ???
Assign( Var("x"), BinOp(Var("x"), "+", Var("y")))
type of variable not part of variable use
Variables: Map
declare-all = alltd(declare)
declare : Param(x, t) -> Param(x, t) with rules( TypeOf : x -> t )
type-of : Var(x) -> t where t := <TypeOf> x
Scope
define page root(x : Int) { action exptest() { for(x : Int in {1,2,x}) { print(x); } }}
multiple occurrences of same identifier corresponding to different declarations
Variables: Map + Renamerename-all = alltd(rename)
rename : Param(x, t) -> Param(y, t) with y := <rename-var>(x, t)
rename-var : (x, t) -> y with y := x{<new>}; rules( TypeOf : y -> t RenameId : x -> y )
rename : Var(x) -> Var(y) where y := <RenameId> x type-of : Var(x) -> t where t := <TypeOf> x
unique annotation
rename occurrences
map variable to type
Term Annotations
t{t1,...,tn}
add additional information to term without affecting signature
Variables: Check
check : e@Var(x) -> (e, $[Variable '[x]' not declared]) where require(<type-of>e)
Variable Binding Constructs
rename : For(x, t, e1, stat1*) -> For(y, t, e2, stat2*) with e2 := <rename-all> e1 with {| RenameId : y := <rename-var>(x, t) ; stat2* := <rename-all> stat1* |}
For defines local variable x in body stat1*’
Editor Interface with Analysis
editor-analyze: (ast, path, fullpath) -> (errors, warnings, notes) with ast2 := <analyze> ast; errors := <collect-all(check, conc)> ast2; warnings := <collect-all(constraint-warning, conc)> ast2; notes := <collect-all(constraint-note, conc)> ast2
analyze = rename-all
Rename, Map, Project, Check
Rename
★ make local variables unique
Map
★ variables to their type
Project
★ compute type of expressions
Check
★ check constraints using types
Reference Resolution
Reference Resolution
Aiding program navigation
★ Hover-click on identifier to jump to declaration
Reuse name resolution infrastructure
editor-resolve: (source, position, ast, path, fullpath) -> target where target := <compute-target> source
Reference Resolution
module nwl-References
imports nwl-References.generated
references
reference _ : editor-resolve
editor-resolve: (SimpleType(type), position, ast, path, fullpath) -> target where Entity(target,_) := <declaration-of> type editor-resolve: (ref@PageRef(x,e*), position, ast, path, fullpath) -> target where TemplateDef(_,target,_,_) := <declaration-of> ref
From Use to Declaration
Experience
Experience
Bootstrapped
★ SDF, Stratego, ATerm, PP
★ Editor service language
Spoofax
★ WebDSL
★ Mobl: mobile web applications
★ Acoda: migration of data models
★ Aster: attribute grammars
★ PIL: platform independent language
★ Student languages
SDF+Stratego
★ Java, AspectJ, XML, PHP, SQL, Jimple, Octave, Dot, ...
Future Work (in progress)
Higher-level definition of scope & type system
★ cover scope systems of real languages
Refactoring
★ layout preservation, generic refactoring strategies
Systematic language design
★ how do we decide that a language design is good?
Interaction design
★ higher-level specifications of interaction
Combining textual and visual
Textual IDE in the browser
- the end -
http://spoofax.org
http://webdsl.org http://mobl.org
Lennart C. L. Kats, Eelco Visser. The Spoofax Language Workbench. Rules for Declarative Specification of Languages and IDEs. OOPSLA 2010
Context-sensitiveTransformation
Template Inliningdefine page blog(b : Blog) { header{output(b.name)} list{ for(p : Post in b.posts) { listitem{ outputPost(p) } } }}define outputPost(pst : Post) { navigate post(pst) { output(pst.name) }}define list() { <ul> elements </ul> }define listitem() { <li> elements </li> }define header() { <h1> elements </h1> }
define page blog ( b : Blog ) { <h1> output(b.name) </h1> <ul> for ( p : Post in b.posts ) { <li> navigate post(p) { output(p.name) } </li> } </ul>}
Context-sensitive Transformation
Local-to-local
★ replace term by another term
Local-to-global
★ local term influence terms in other parts of model
Global-to-local
★ global information influences transformation of local terms
define page blog(b : Blog) { header{output(b.name)} list{ for(p : Post in b.posts) { listitem{ outputPost(p) } } }}define outputPost(pst : Post) { navigate post(pst) { output(pst.name) }}define list() { <ul> elements </ul> }define listitem() { <li> elements </li> }define header() { <h1> elements </h1> }
define page blog ( b : Blog ) { <h1> output(b.name) </h1> <ul> for ( p : Post in b.posts ) { <li> navigate post(p) { output(p.name) } </li> } </ul>}
Inlining is Global-to-Local
outputPost(p) -> navigate post(p) { output(p.name) }where define outputPost(pst : Post) { navigate post(pst) { output(pst.name) } }
Inlining as Rewrite Problem
(this is not a valid Stratego rewrite rule)
Rewrite Rules are Context-free
desugar : Property(x, t) -> Property(x, t, [])
Simplification: Parameterless Templates
define page blog(b : Blog) { ... footer()}define footer() { navigate url(http://webdsl.org) { “WebDSL” }}
define page blog ( b : Blog ) { ... container() { navigate url(http://webdsl.org) { “WebDSL” } }}
Rewrite in Context
Inline : [def@TemplateDef(f,[],elem1*) | def1*] -> [def | def2*] where def2* := <alltd((Call(f) -> Call("container", elem1*)))> def1*
bound in contextlocal traversal
container needed to replace single call with multiple elements
Intermezzo:Rewriting Strategies
transformation
is
partial* function from terms to terms
* transformation may fail to apply to a term
Defining Transformations: Rules & Strategies
Rewrite rules are basic transformations
★ transform term matching lhs to instantiation of rhs
★ evaluate condition (where clause)
Strategies combine rules into complex transformations
★ select rules to apply
★ select algorithm to apply rules
Strategy combinators
★ composition of custom transformations
Strategy Combinators
id★ Identity
fail★ failure
s1 ; s2★ sequential composition
s1 <+ s2★ choice
Variables
{x, y : s}★ term variable scope
?t★ match term pattern t
!t★ build term pattern t
t1 := t2★ match term t2 to term (pattern) t1
<s> t★ apply strategy s to term t
Rewrite Rules
l : t1 -> t2 where s★ named, scoped rewrite rule
★ all variables in t1, t2, s are in scope of the rule
(t1 -> t2 where s)★ unscoped rewrite rule
★ variables are in scope of enclosing scope
Rewrite in Context
Inline : [def@TemplateDef(f,[],elem1*) | def1*] -> [def | def2*] where def2* := <alltd((Call(f) -> Call("container", elem1*)))> def1*
bound in context
[ TemplateDef( "footer" , [] , [Navigate(PageRef("webdsl", []), [String("WebDSL")])] ), TemplateDef( "blog" , [Param("b", SimpleType("Blog"))] , [Call("footer")] )]
Inline : [def@TemplateDef(f,[],elem*) | def1*] -> [def | def2*] where def2* := <alltd((Call(f) -> Call("container", elem*)))> def1*
f elem*
def1*Call(f)
Strategy Definitions
f(x,y|a,b) = s★ x, y are strategy parameters
★ a, b are term parameters
★ s uses all parameters
f(x) = s f = s★ term parameters are optional
★ all parameters are optional
Examples
★ try(s) = s <+ id★ repeat(s) = try(s; repeat(s))
Rules with Parameters
Transform all elements of a list
Invert order of elements of a list
Pair elements of two lists
inverse(|ys) : [] -> ys
inverse(|ys) : [x|xs] -> <inverse(|[x|ys])> xs
map(s) : [] -> []
map(s) : [x|xs] -> [<s>x | <map(s)> xs]
zip(s) : ([],[]) -> []
zip(s) : ([x|xs],[y|ys]) -> [<s>(x,y) | <zip(s)>(xs,ys)]
Traversal Combinators
all(s)★ apply s to all direct sub-terms (children)
one(s)★ apply s to exactly one sub-term
some(s)★ apply s to at least one sub-term
Traversal Strategies
topdown(s) = s; all(topdown(s))★ apply s to all sub-terms in top-down order
bottomup(s) = all(bottomup(s)); s★ apply s to all sub-terms in bottom-up order
oncetd(s) = s <+ one(oncetd(s))★ apply s to one sub-term
alltd(s) = s <+ all(alltd(s))★ apply s to frontier
Rewrite in Context: Local Traversal
Inline : [def@TemplateDef(f,[],elem1*) | def1*] -> [def | def2*] where def2* := <alltd((Call(f) -> Call("container", elem1*)))> def1*
local traversal
Dynamic Rewrite Rules
Rewrite in Context: Not Optimal
requires def before use local traversal for each declaration
Inline : [def@TemplateDef(f,[],elem1*) | def1*] -> [def | def2*] where def2* := <alltd((Call(f) -> Call("container", elem1*)))> def1*
Dynamic Rewrite Rules
separate traversal from rule definition (binding closures)
declare-inline : TemplateDef(f,[],elem1*) -> TemplateDef(f,[],elem1*) where rules( InlineTemplate : Call(f) -> Call("container", elem1*) )
inline = alltd(declare-inline); topdown(try(InlineTemplate))
Inline : [def@TemplateDef(f,[],elem1*) | def1*] -> [def | def2*] where def2* := <alltd((Call(f) -> Call("container", elem1*)))> def1*
outputPost(p) -> navigate post(p) { output(p.name) }where define outputPost(pst : Post) { navigate post(pst) { output(pst.name) } }
Inlining as Rewrite Problem (Revisited)
(informal)
declare-inline : TemplateDef(f,[],elem1*) -> TemplateDef(f,[],elem1*) where rules( InlineTemplate : Call(f) -> Call("container", elem1*) )
(formal; but not yet complete)
Template Parameterdefine page blog(b : Blog) { header{output(b.name)} list{ for(p : Post in b.posts) { listitem{ outputPost(p) } } }}define outputPost(pst : Post) { navigate post(pst) { output(pst.name) }}define list() { <ul> elements </ul> }define listitem() { <li> elements </li> }define header() { <h1> elements </h1> }
define page blog ( b : Blog ) { <h1> output(b.name) </h1> <ul> for ( p : Post in b.posts ) { <li> navigate post(p) { output(p.name) } </li> } </ul>}
declare-inline : def@TemplateDef(mod*,f,param*,elem1*) -> def where rules( InlineTemplate : Call(f, e*, []) -> Call("container", [], elem3*) where elem3* := <substitute> (param*, e*, elem1*) ) substitute : (param*, e*, elem1*) -> elem2* where {| Subst : <zip(bind-arg)> (param*, e*) ; elem2* := <alltd(Subst)> elem1* |} bind-arg : (Param(x, t), e) -> (Param(x, t), e) where rules( Subst : Var(x) -> e )
Inlining Templates with Parameters
declare-inline : def@TemplateDef(mod*,f,param*,elem1*) -> def where rules( InlineTemplate : Call(f, e*, elem2*) -> Call("container", [], elem3*) where {| Subst : rules( Subst : Elements() -> Call("container",[],elem2*) ) ; elem3* := <substitute> (param*, e*, elem1*) |} )
define list() { <ul> elements </ul> }
Element Parameters
Removing Intermediate Structures
rules // remove containers desugar-container : [Call("container",[], elem1*) | elem2*] -> [elem1*, elem2*] desugar : elem1* -> elem2* where elem2* := <at-suffix(desugar-container)> elem1*
container needed to replace single call with multiple elements
Inlining Strategy
module template-inlining
imports libstratego-libimports include/nwlimports desugar
strategies inline-all = desugar-all; alltd(declare-inline); innermost(desugar <+ InlineTemplate)
rules declare-inline : ...
module template-inlining
imports libstratego-libimports include/nwlimports desugar
strategies inline-all = desugar-all; alltd(declare-inline); innermost(desugar <+ InlineTemplate)
rules declare-inline : TemplateDef(mod*,f,param*,elem1*) -> TemplateDef(mod*,f,param*,elem1*) where rules( InlineTemplate : Call(f, e*, elem2*) -> Call("container", [], elem3*) where {| Subst : rules( Subst : Elements() -> Call("container", [], elem2*) ) ; elem3* := <substitute> (param*, e*, elem1*) |} ) substitute : (param*, e*, elem1*) -> elem2* where {| Subst : <zip(bind-arg)> (param*, e*) ; elem2* := <alltd(Subst)> elem1* |} bind-arg : (Param(x, t), e) -> (Param(x, t), e) where rules( Subst : Var(x) -> e )
rules // remove containers desugar-container : [Call("container",[], elem1*) | elem2*] -> [elem1*, elem2*] desugar : elem1* -> elem2* where elem2* := <at-suffix(desugar-container)> elem1*
Template Inlining Transformation