mastering grammars with petitparser
DESCRIPTION
PetitParser is a dynamic parser framework combining ideas from scannerless parsing, parser combinators, parsing expression grammars and packrat parsers. In this hands-on session we learn how to build simple parsers and how to model, test, compose and reuse complex grammars. Additionally, we will look at some tools and the reflective facilities provided by the PetitParser framework. Basic knowledge of the Smalltalk programming language is a requirement. Bring your laptop to reproduce the examples and solve some simple tasks.TRANSCRIPT
![Page 1: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/1.jpg)
Mastering Grammarswith
Lukas Renggli
![Page 3: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/3.jpg)
Agenda
1. PetitParser in a Nutshell
2. Combinatorial Parsing
3. Complex Grammars
4. Advanced Use
![Page 4: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/4.jpg)
1in a Nutshell
![Page 5: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/5.jpg)
a..z a..z
0..9
ID ::= letter { letter | digit } ;
![Page 6: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/6.jpg)
letter
letter digit
sequence
choice
many
ID ::= letter { letter | digit } ;
![Page 7: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/7.jpg)
letter
letter digit
sequence
choice
many
id := #letter asParser , (#letter asParser / #digit asParser) star
![Page 8: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/8.jpg)
id parse: 'yeah'. " -‐-‐> #($y #($e $a $h)) "
id parse: 'f12'. " -‐-‐> #($f #($1 $2)) "
id parse: '123'. " -‐-‐> letter expected at 0 "
![Page 9: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/9.jpg)
‣ Test the identifier parser
‣ Inspect the parser object
‣ Create and test an integer parser
Example 1
![Page 10: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/10.jpg)
2Combinatorial Parsing
![Page 11: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/11.jpg)
Parser Terminals
$a asParser letter ‘a’
'abc' asParser string ‘abc’
#any asParser any character
#digit asParser the digits 0..9
#letter asParser the letters a..z and A..Z
nil asParser the empty parser
These factory methods are defined inPPPredicateObjectParser class
![Page 12: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/12.jpg)
Parser Operators
p1 , p2 sequence
p1 / p2 ordered choice
p star zero-or-more (0..*)
p plus one-or-more (1..*)
p optional zero-or-one (0..1)
p1 separatedBy: p2
p1 delimitedBy: p2
see the operations protocols in PPParser
for more operators
![Page 13: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/13.jpg)
Parser Predicates
p and conjunction (non-consuming look-ahead)
p not negation (non-consuming look-ahead)
p end end of input
![Page 14: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/14.jpg)
Parser Actions
p ==> [ :arg | ] transformation
p flatten create string
p token create token
p trim trim whitespaces
see the operations-mapping protocol in PPParser
for more actions
![Page 15: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/15.jpg)
term ::= prod "+" term | prod ;
prod ::= prim "*" prod | prim ;
prim ::= "(" term ")" | number ;
number ::= "0" .. "9" ;
![Page 16: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/16.jpg)
number := #digit asParser plus flatten trim ==> [ :string | string asNumber ].
term := PPUnresolvedParser new.prod := PPUnresolvedParser new.prim := PPUnresolvedParser new. term def: (prod , $+ asParser trim , term ==> [ :nodes | nodes first + nodes last ]) / prod.prod def: (prim , $* asParser trim , prod ==> [ :nodes | nodes first * nodes last ]) / prim.prim def: ($( asParser trim , term , $) asParser trim ==> [ :nodes | nodes second ]) / number.
start := term end.
![Page 17: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/17.jpg)
start parse: '1 + 2 * 3'. " -‐-‐> 7 "
start parse: '(1 + 2) * 3'. " -‐-‐> 9 "
![Page 18: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/18.jpg)
‣ Add support for negative numbers
‣ Add support for floating point numbers
‣ Add support for subtraction & division
Example 2
![Page 19: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/19.jpg)
3Complex Grammars
![Page 20: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/20.jpg)
PetitParser Scripts
![Page 21: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/21.jpg)
PetitParser Scripts
quick to
write
embed into Smalltalk
![Page 22: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/22.jpg)
PetitParser Scripts
quick to
write
embed into Smalltalkhard
to test
difficultto reuse
messy if large
![Page 23: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/23.jpg)
start
PPCompositeParser
![Page 24: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/24.jpg)
starttermprod...
start
PPCompositeParser
ExpressionGrammar
![Page 25: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/25.jpg)
starttermprod...
start
PPCompositeParser
ExpressionGrammar
start ^ term end
One Method per Production
![Page 26: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/26.jpg)
starttermprod...
start
PPCompositeParser
ExpressionGrammar
One Instance Variable per Production
term ^ (prod , $+ asParser trim , term) / prod
![Page 27: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/27.jpg)
starttermprod...
start
PPCompositeParser
ExpressionGrammar
Refer to Productions by Inst-Var Reference
prod ^ (prim , $* asParser trim , term) / prim
![Page 28: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/28.jpg)
starttermprod...
start
PPCompositeParser
ExpressionGrammar
rest ismagic
![Page 29: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/29.jpg)
termprodnumber...
ExpressionEvaluator
starttermprod...
ExpressionGrammar
![Page 30: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/30.jpg)
‣ Implement an expression grammar
‣ Implement an expression evaluator
‣ Implement an expression pretty printer
Example 3
![Page 31: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/31.jpg)
4Advanced Use
![Page 32: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/32.jpg)
does not justwork on Strings
![Page 33: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/33.jpg)
Matching
p matches: 'abc'.
p matchesIn: 'abc'.
p matchesIn: 'abc' do: [ :each | ].
p matchingRangesIn: 'abc'.
p matchingRangesIn: 'abc' do: [ :interval | ].
![Page 34: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/34.jpg)
GUI PPBrowser openWorld ➔ Tools ➔ PetitParser
![Page 35: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/35.jpg)
Reflection
p allParser.
p allParserDo: [ :each | ].
p firstSet.
p followSet.
p cycleSet.
![Page 36: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/36.jpg)
Transformations
p replace: p1 with: p2.
p transform: [ :parser | ].
Like #collect: on Collection, but transforms the whole grammar graph.
![Page 37: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/37.jpg)
Pattern Searching
found := PPSearcher new
matches: PPPattern any star plus
do: [ :parser :answer | parser ];
execute: p initialAnswer: nil
A placeholder matching any parser
Grammar tobe searched
![Page 38: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/38.jpg)
Pattern Rewriting
pattern := PPPattern any.
rewritten := PPRewriter new
replace: pattern star plus
with: pattern star;
execute: p
Grammar tobe rewritten
Same pattern used in search and replace.
![Page 39: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/39.jpg)
Grammar Optimization
fast := slow optimize
Many behavior preserving rewrite rules applied for you.
![Page 40: Mastering Grammars with PetitParser](https://reader034.vdocument.in/reader034/viewer/2022052321/549c6d41ac7959e22a8b4724/html5/thumbnails/40.jpg)
0
1000000
2000000
3000000
LALR PetitParser Hand-Written
char
s/se
c
Old VM Cog VM