transform your state \/ err

@gerferra

Transform your State \/ Err

An example of functional parsing with state and error handling

@gerferra

Initial language

Arithmetic expressions over integers

3*3+4*4*(1+2);

Functional parsing

Write a parser using “just functions” ...

… within some programming language

… in the form of some embedded DSL

Functional parsingAll of this has a proper name:

Parser combinators

A collection of basic parsing functions that recognise a piece of input

A collection of combinators that build new parsers out of existing oneshttp://www.fing.edu.uy/inco/cursos/pfa/uploads/Main/parsercombinators

Parser combinatorsScala has many parsing combinator libraries

Two common ones:

scala-parser-combinators

parboiled2

Two common ones:

parboiled2

The one we will be using

Two common ones:

parboiled2 Not really parser combinators ...

but very fast!

Two common ones:

parboiled2

New kid on the block: lihaoyi/fastparse

Part of the Scala standard library but distributed separately

libraryDependencies += "org.scala-lang.modules" %% "scala-parser-combinators" % "1.0.4"

Slow but good enough in this case

The first (?) parser combinator library for Scala

Many (but not all) of the other parser combinator libraries use similar names/symbols for the combinators

scala-parser-combinatorsimport scala.util.parsing.combinator._

object BasicCombinators extends Parsers

with RegexParsers {

def pA : Parser[Char] = 'A' def pB : Parser[Char] = 'B' def pAlt: Parser[Char] = pA | pB def pSeq: Parser[Char ~ Char] = pA ~ pB def pRep: Parser[List[Char]] = rep(pA) def pOpt: Parser[Option[Char]] = opt(pA)}

import scala.util.parsing.combinator._

with RegexParsers {

scala-parser-combinatorsBase type

with RegexParsers {

Useful extensions

with RegexParsers {

alternative, sequence, repetition, optionals ...

trait Parsers { type Input

sealed trait ParseResult[+T]

trait Parser[+T] extends (Input => ParseResult[T])}

Success[+T] | NoSuccess...plus the input not consumed

~ Input => (T, Input)

trait Parser[+T] extends (Input => ParseResult[T])} A Parser is a function taking some input and

returning either some value of type T, or a failure

trait Parser[+T] extends (Input => ParseResult[T])} A Parser is a function taking some input and

returning either some value of type T, or a failure

Returning to our language

3*3+4*4*(1+2);

Returning to our languageobject Parser1 extends RegexParsers {

def pL1: Parser[List[Int]] = rep(pExpr <~ ";")

def pExpr = pAdd

def pAdd: Parser[Int] = pProd ~ "+" ~ pAdd ^^ { case left ~ _ ~ right => left + right } | pProd

def pProd: Parser[Int] = pAtom ~ "*" ~ pProd ^^ { case left ~ _ ~ right => left * right } | pAtom

def pAtom: Parser[Int] = """\d+""".r ^^ (_.toInt) | "(" ~> pExpr <~ ")"}

def pExpr = pAdd

Many things going on

def pExpr = pAdd

We are evaluating the result directly, avoiding construct an AST

def pExpr = pAdd

Precedence rules are encoded in the grammar

def pExpr = pAdd

This is evaluating from right to left.

If you need to control associativity, the rep combinator is your friend.

def pExpr = pAdd

Regular expressions are converted to Parsers implicitly

def pExpr = pAdd

^^ is an alias of the map function

def pExpr = pAdd

Allows to convert the Parser[String] to Parser[Int]^^ is an alias of

the map function

def pExpr = pAdd

Allows to convert the Parser[String] to Parser[Int]

String => Int

^^ is an alias of the map function

def pExpr = pAdd

Parse two inputs in sequence discarding the input of the left or the right respectively

def pExpr = pAdd

Pattern matching anonymous function

def pExpr = pAdd

Allows to deconstruct the result of this parser

def pExpr = pAdd

Allows to deconstruct the result of this parser

Int IntString

“+” discarded Int

def pL1: Parser[List[Int]] = rep(pExpr <~ ";") def pExpr = pAdd

We want a list of all the expressions we encounter separated by “;”

def pL1: Parser[List[Int]] = rep(pExpr <~ ";") def pExpr = pAdd

All good?

Mini testimport Parser1._

parse(pExpr, "1") //> [1.2] parsed: 1parse(pExpr, "2+2") //> [1.4] parsed: 4parse(pExpr, "3*3+4*4*(1+2)") //> [1.14] parsed: 57

parse(pL1, """1; |2+2; |3*3+4*4*(1+2);""".stripMargin) //> [3.15] parsed: List(1, 4, 57)

parse(pL1, "1; 2+2; 3*3+4*4*(1+2);") //> [1.8] parsed: List(1, 4, 57)

Now …

What if we want to add variables?

We need to handle two more cases:

Assignation of variablesReferences to variables

def pRef = pName ^^ { x => get(symtab, x)}

def pVar = "var" ~ pName ~ "=" ~ pExpr ^^ { case _ ~ name ~ _ ~ expr => put(symtab, name, expr) }

And we need to carry a symbol table ...

?def pRef = pName ^^ { x => get(symtab, x)}

Carrying a symbol table

On pRef we need to access the symbol table but leave it unmodified

On pVar we need to update the symbol table

Otherwise we just need to pass it untouched

def pRef = pName ^^ (x => get(symtab, x))

Enter scalaz.State[S,A]

scalaz has a type that represents state transitions

State[S,A] wraps functions with type:

S => (S, A)Current state

New stateValue computed

We need to change the type of our parsers ...

Parser[A]

Parser[State[S,A]]

Parser[State[SymTab, A]]

Parser[S[A]]

We need to change the type of our parsers ...We now return A, but in the context of some stateful computationParser[A]

Parser[State[S,A]]

Parser[S[A]]

We need to change the type of our parsers ...We now return A, but in the context of some stateful computation

Our state is the symbol table~Map[String, Int]

Parser[A]

Parser[State[S,A]]

Parser[S[A]]

Parser[A]

Parser[State[S,A]]

Parser[S[A]]

type S[A] = State[SymTab,A]

Parser[A]

Parser[State[S,A]]

Parser[S[A]]

type S[A] = State[SymTab,A]

type P[A] = Parser[S[A]]

… and change the parsers themselves a bit

… and change the parsers themselves a bitdef pAdd: P[Int] = pProd ~ "+" ~ pAdd ^^ { case leftS ~ _ ~ rightS => for { left <- leftS right <- rightS } yield { left + right } } | pProd

P[Int] == Parser[S[Int]]

S[Int]

leftS and rightS areS[Int] and can’t be added directly

S[Int]

But State[S,A] is a Monad[A]

State[SymTab,A]

S[Int]

State[SymTab,A]So we can use a for-comprehension to work with the ints

S[Int]

State[SymTab,A]When we treatState[S,A] as a Monad[A], we work with the values, and the state transitions happens in the background

S[Int]

State[SymTab,A]Here we are chaining three state transitionss0 => (s1, left)s1 => (s2, right)s2 => (s2, left+right)

S[Int]

State[SymTab,A]Which can be seen as one big transition:

s0 => ... (s2, left+right)

The State[S,A] we return

… and change the parsers themselves a bitdef pRef: Parser[Int] = pName ^^ { x => get(symtab, x) }

def pRef: P[Int] = pName ^^ { name => for { symtab <- State.get } yield { symtab(name) } }

Here we need to access to the symbol table—our state

The trick of State.get is to create a new State[S,S] where the state is used as the value

Wraps a state transition { s => (s, s) }

With such State[S,S] created, now we can use again a for-comprehension to work with the symbol table (the value)

We are chaining two state transitions:s => (s, s)s => (s, s(name))

… and change the parsers themselves a bit

def pRef: Parser[Int] = pName ^^ { x => get(symtab, x) }

Which can be seen as:

s => ... (s, s(name))

The State[SymTab, Int] == S[Int] we return

… and change the parsers themselves a bitdef pVar: Parser[Unit] = "var"~pName~"="~pExpr ^^ { case _ ~ name ~ _ ~ expr => put(symtab,name,expr) }

def pVar: P[Unit] = "var"~pName~"="~pExpr ^^ { case _ ~ name ~ _ ~ exprS => for { expr <- exprS symtab <- State.get unit <-State.put( symtab+(name->expr) ) } yield { unit } }

The last case

We need to modify the state

S[Int]

S[SymTab]S[Int]

To change the state we use State.put

S[SymTab]S[Int]

Given s, it creates aState[S,Unit] representing this state transition: { _ => (s, ()) }

S[SymTab]S[Int]

This discards any previous state in favor of the one provided, and returns () { _ => (s, ()) }

S[SymTab]S[Int]

So, the value we get in the for-comprehension is the unit

S[Unit]

S[SymTab]S[Int]

We have four state transitions:s0 => (s1, expr)s1 => (s1, s1)_ => (s2=s1+(name->expr), ())s2 => (s2, ())

S[Unit]

S[SymTab]S[Int]

So we are returning:

s0 => … (s1+(name->expr), ())

State[SymTab, Unit] == S[Unit] S[Unit]

S[SymTab]S[Int]

Now we know the basics of the State monad

Running the stateimport Parser2._

parse(pExpr, "1") //> [1.2] parsed: scalaz.package$State$$anon$3@7f560810

Our parser now returns State[SymTab,A], not directly a value

To compute the final value, we need to provide an initial state and then all the state transitions can be run

parse(pExpr, "1").map(state => state.run(Map.empty))//> [1.2] parsed: (Map(),1)

map over ParseResult[S[Int]]

map over ParseResult[S[Int]]Initial state

We obtain the final state and the value computed

parse(pExpr, "1").map(_.run(Map.empty))//> [1.2] parsed: (Map(),1)

parse(pVar, "var x = 1").map(_.run(Map.empty))//> [1.10] parsed: (Map(x -> 1),())

parse(pExpr, "y * 3").map(_.run(Map("y" -> 2)))//> [1.6] parsed: (Map(y -> 2),6)

The final state reflects the binding of 1 to x

We can provide bindings in the initial state

Running the stateparse(pL2, """var x = 1; |var y = x * 3; |x + y; |(x + 1) * 3 + 1; |var z = 8;""".stripMargin).map(_.run(Map.empty))//> [5.11] parsed: (Map(x -> 1, y -> 3, z -> 8),List(4, 7))

Now ...

What if we reference an undeclared variable?parse(pL2, "var x = y;").map(_.run(Map.empty))

Now ...

What if we reference an undeclared variable?parse(pL2, "var x = y;").map(_.run(Map.empty))

Exception in thread "main" java.util.NoSuchElementException: key not found: y

at scala.collection.MapLike$class.default(MapLike.scala:228)at scala.collection.AbstractMap.default(Map.scala:59)at scala.collection.MapLike$class.apply(MapLike.scala:141)at scala.collection.AbstractMap.apply(Map.scala:59)...

Handling errors

If our computation can fail, then our types are wrong. We can’t just return A

Handling errors

If our computation can fail, then our types are wrong. We can’t just return A

We need to handleReferences to undeclared variablesDeclarations of already declared variables

scalaz.\/

\/[A,B] == A\/B, called “or”, “xor”, “either”, “disjoint union”,..., can be used to handle the cases when a computation can fail

A \/ BThe type of the error. By convention on the left

The type of the value we are computing. By convention on the right

scalaz.\/

\/[A,B] is the scalaz version ofscala.Either[A,B]

Handling errors

We will have to change our types again:

type V[A] = Err \/ A

type S[A] = State[SymTab, V[A]]

sealed abstract class Err(override val toString: String)

case class NotDeclared(name: String) extends Err(s"`$name' not declared.")

case class AlreadyDeclared(name: String, value: Int) extends Err(s"`$name' already declared with value `$value'.")

Bad news

It gets ugly ...

It gets ugly ...def pVar: P[Unit] = "var" ~ pName ~ "=" ~ pExpr ^^ { case _ ~ name ~ _ ~ exprS => for { exprV <- exprS symtab <- State.get[SymTab] newSymTab = for { expr <- exprV s <- symtab.get(name) match { // if the name exists, creates a left with the error case Some(v) => AlreadyDeclared(name, v).left // if not exists, returns the symtab as a right case None => symtab.right } } yield { s + (name -> expr) // only executes if everything is right } res <- newSymTab match { // if the newSymTab is a left, I need to put the error in the State case -\/(l) => State.state[SymTab, V[Unit]](l.left) // if the newSymTab is a right, I need to use it as the new state, and put the () inside a \/ case \/-(r) => State.put(r).map(_.right) } } yield { res }}

This is just one parser

exprS now contains a \/[Err,Int]

I’m using an extra for-comprehension to deal with it

Some juggling to construct a left value if the symbol table already has the name

If either exprV or this contains a left, all the for-comprehension will be a left, ie, newSymTab will be a left

Here I’m “folding” over the\/[Err, SymTab]. If is a left, I need to put that inside the State. If is a right, I need to use the value as the new state

-\/ is the extractor of left values, \/- is the extractor of right values

State.state allows to put a value inside a State[S,A]. For a given `a’, represents a transition { s => (s, a) }

Good news

Monad transformers

The only thing I know about them, is that they let you combine the effects of two monads, ...

Monad transformers

… scalaz has a monad transformer, StateT, that let you combine the State monad with any other monad, …

Monad transformers

… scalaz has a monad transformer, StateT, that let you combine the State monad with any other monad, …

… and this solves all our problems

Combining State and \/

This are our final type aliases

type SymTab = Map[String, Int]

type S[A] = StateT[V, SymTab, A]

This is the only new one

StateT is parameterized on the monad with which is combined

Here is using the monad of\/[A,B]

And one final ingredientval State = StateT.stateTMonadState[SymTab, V]

The companion object of StateT does not have the functions get, put, state, etc.

Instead we have to instantiate one of this “beasts”

Combining State and \/With that changes, our parser works as before without any modification

Combining State and \/import Parser4._

parse(pExpr, "1").map(_.run(Map.empty))//> [1.2] parsed: \/-((Map(),1))

parse(pVar, "var x = 1").map(_.run(Map.empty))//> [1.10] parsed: \/-((Map(x -> 1),()))

parse(pExpr, "y * 3").map(_.run(Map("y" -> 2)))//> [1.6] parsed: \/-((Map(y -> 2),6))

Combining State and \/parse(pL4, """var x = 1; |var y = x * 3; |x + y; |(x + 1) * 3 + 1; |var z = 8;""".stripMargin).map(_.run(Map.empty))//> [5.11] parsed: \/-((Map(x -> 1, y -> 3, z -> 8),List(4, 7)))

The only difference is that now the results are inside a \/, in particular a \/-, ie, a right value, which indicates that everything is OK.

Handling errors

Everything else is working the same, even our exceptions when an undeclared variable is referenced

Handling errors

Everything else is working the same, even our exceptions when an undeclared variable is referenced

We must change pRef y pVar to handle the error cases

Handling errorsdef pRef: P[Int] = pName ^^ { name => for { symtab <- State.get } yield { symtab(name) } }

def pRef: P[Int] = pName ^^ { name => for { symtab <- State.get vErrInt = symtab.get(name) .toRightDisjunction( NotDeclared(name)) res <- StateT[V, SymTab, Int] { s => vErrInt.map(s -> _) } } yield { res } }

We are converting an Option[Int] into \/[Err,Int]

We construct our StateT “by hand” by passing the function directly

This StateT wraps functions of type SymTab => V[(SymTab,A)] == SymTab => \/[Err, (SymTab,A)]

vErrInt is of type\/[Err, Int], we convert it to\/[Err, (SymTab, Int)] with map

This state transition will only be made if vErrInt is a right value. If not, all the computation will return the left value

Here is happening the combination of both monad effects

Handling errorsdef pVar: P[Unit] = "var" ~ pName ~ "=" ~ pExpr ^^ { case _ ~ name ~ _ ~ exprS => for { expr <- exprS symtab <- State.get unit <- symtab.get(name) match { case Some(v) => StateT[V, SymTab, Unit] { s => AlreadyDeclared(name, v).left } case None => State.put(symtab + (name->expr)) } } yield { unit }}

def pVar: P[Unit] = "var"~pName~"="~pExpr ^^ { case _ ~ name ~ _ ~ exprS => for { expr <- exprS symtab <- State.get unit <-State.put( symtab+(name->expr) ) } yield { unit } } We check if the declaration

already exists

def pVar: P[Unit] = "var"~pName~"="~pExpr ^^ { case _ ~ name ~ _ ~ exprS => for { expr <- exprS symtab <- State.get unit <-State.put( symtab+(name->expr) ) } yield { unit } } If exists, we construct a StateT with

a function returning a left. This stops any further state transitions

def pVar: P[Unit] = "var"~pName~"="~pExpr ^^ { case _ ~ name ~ _ ~ exprS => for { expr <- exprS symtab <- State.get unit <-State.put( symtab+(name->expr) ) } yield { unit } } If the value not exists, we proceed as

before, updating the state

def pVar: P[Unit] = "var"~pName~"="~pExpr ^^ { case _ ~ name ~ _ ~ exprS => for { expr <- exprS symtab <- State.get unit <-State.put( symtab+(name->expr) ) } yield { unit } } In both cases we are returning

StateT[V, SymTab, Unit]

Now errors are handled properlyimport Parser4._

parse(pL4, "var x = y;").map(_.run(Map.empty))//> [1.11] parsed: -\/(`y' not declared.)

parse(pL4, "var x = 1; x * x; var x = 2;").map(_.run(Map.empty))//> [1.29] parsed: -\/(`x' already declared with value `1'.)

Now errors are handled properlyimport Parser4._

parse(pL4, "var x = y;").map(_.run(Map.empty))//> [1.11] parsed: -\/(`y' not declared.)

parse(pL4, "var x = 1; x * x; var x = 2;").map(_.run(Map.empty))//> [1.29] parsed: -\/(`x' already declared with value `1'.)

Both return left values

github.com/gerferra/parser-stateT

transform your state \/ err

Software

hs err pid3407

s. i err f edu · 2015. 3. 9. · 2013-14 s. i err f edu...

je 'wiyl il je err err je rn - society for industrial ... je...

warm-up err… bellringer …err…do now

orphannuclearreceptorestrogen-relatedreceptor (err ... ·...

to err is human; not to err is better! vaccination errors...

solvang city hall n yh edd err h edd err

measurement of call drops due to err ind in stable state

jon nistor nistor@torix.ca boulevard of broken networks ·...

err 2013 - 6

reas'ningbut 70 err - butler.edu

メトロイド - 任天堂ホームページ...disk err.oi...

man 8040t err

citroen err table

apprenticeship workbook err type

err documentation

len wildphase err phase1 - arxiv · tracted from the...

man 8055m err

discrete fourier transform (dft) - iowa state university

sear led err ida