specs2

Specs2Library for writing Acceptance And

Unit Tests

Specs2Library for writing Acceptance And

Unit Tests

Piyush MishraSoftware Consultant

Knoldus Software LLP

Piyush MishraSoftware Consultant

Knoldus Software LLP

Topics CoveredTopics Covered

What is Specs2

The design principles of specs2

Unit Specifications

Acceptance Specifications

Matchers

Runners

What is Specs2


Unit Specifications

Acceptance Specifications

Matchers

Runners

What is Specs2What is Specs2

Specs is a DSL in Scala for doing BDD (Behavior-Driven -Development).

Design Principles of Specs2Design Principles of Specs2

Do not use mutable variables Use a simple structure Control the dependencies (no cycles) Control the scope of implicits


Guide to write Unit SpecificationsGuide to write Unit Specifications

Unit specifications

Extend the org.specs2.mutable.Specification trait are mutable

Use should / in format in creates an Example object containing a Result should creates a group of Example objects

Unit specifications



Creating Unit SpecificationsCreating Unit Specifications

specs2

the remix, now complete with user Q & A

What's new in the Scala BDD world?http://specs2.org

Rose Toomeyny-scala @ 24 May 2011Revised 6 June with assistance from Eric Torreborre

Been here before? Your key to the remix

The existing presentation has been marked up for your convenience:

This is new content.

This content was wrong. So please delete it from your memory banks.

This is a correction.

This is a clarification.

This is what Eric Torreborre said.specs2: State of the art executable software specifications

The evolution of specs2 Availability

Field guide to specs2 acceptance specs unit specs Explanation of thrown expectations

Migrating to specs2 Test case: migrating Salat Configuring specs2

Cool new features of specs2 JSON matchers Specs2 matchers in the wild Contexts Scalacheck Online Resources User Q & A

The evolution of specs2

specs is a DSL in Scala for doing BDD (Behaviour-Driven Development).

specs2 is a complete rewrite of specs 1.x.



Eric Torreborre, the creator of specs and specs2, will be giving a presentation on the design philosophy of specs2 at Functional Programming Sydney in July.

Follow @specs2.org on Twitter for details!Availabilityspecs2

Eric:

specs2 will not be available for Scala 2.7.7 because it depends on named parameters.

I have to maintain a separate branch for 2.8.x and 2.9.x so SNAPSHOTs will be more frequent on 2.9.0-1.

specs2 is available for Scala 2.8.0, 2.8.1, 2.9.0 and 2.9.0-1.specs

specs 1.6.x is available for Scala 2.7.7, 2.8.1 and 2.9.0 at scala-tools.Field Guide to specs2Unit specifications



Acceptance specs

Extend the org.specs2.Specification trait are functional when extending the default org.specs2.Specification trait

Must define a method called is that takes a Fragments object, which is composed of: an optional SpecStart a list of Fragment objects an options SpecEnd

Both types of specifications contain a list of specification fragments provided by the is method in the SpecificationStructure trait.What is a specification fragment?

Simple text to describe the test case

Examples

a description and executable code that returns a Result, such as a standard result (success, failure) a matcher result a boolean value

Steps and actions that return success or failure reported only if an exception occurs SpecStart and SpecEnd delimiters in acceptance specs tagging fragments to define which fragments should be included or excluded

formatting fragments such as line breaks and tabs to make the test output pleasing to the eye

Execution

specs2 executes examples concurrently by default. You have to explicitly specify when you need sequential execution.

Fragments are sorted in groups so that all the elements of the group can be executed concurrently.

As each group of Example fragments runs concurrently, each Result is collected in a sequence of ExecutedFragments, which are then reduced for reporting.

Step can be used to break up the sequences in order to do some intitialisation or cleanup.What is a result?

An instance of org.specs2.executable.Result contains:

a message describing the outcome a message describing the expectation(s)

StandardResults indicate the result of executing an example:

success failure anError pending skipped

A MatcherResult contains the outcome of an expectation, such as

"hello" must contain("lo") // success"hello" must contain("lz") // failure

Creating a unit specification

package prasinous.unit

import org.specs2.mutable._import prasinous._

class TrivialUnitSpec extends Specification {

"String reverser" should { "reverse String" in { StringReverser("hello") must_== "olleh" StringReverser("") must beEmpty StringReverser(null) must beNull } }

"Option string reverser" should { "reverse Option[String]" in { OptionStringReverser(Some("hello")) must beSome("olleh") // it's in the spec, that means i expected it, right? OptionStringReverser(Some(null)) must beSome(null) OptionStringReverser(None) must beNone } }

}

Running a unit specification

Fire up sbt and run

> test-only prasinous.unit.TrivialUnitSpec

[info] == prasinous.unit.TrivialUnitSpec ==[info] String reverser should[info] + reverse String[info][info] Option string reverser should[info] + reverse Option[String][info][info][info] Total for specification TrivialUnitSpec[info] Finished in 72 ms[info] 2 examples, 0 failure, 0 error[info][info] == prasinous.unit.TrivialUnitSpec ==

Creating an acceptance specification

package prasinous.acceptance

import org.specs2._import prasinous._

class TrivialAcceptanceSpec extends Specification { def is =

"This is a specification to check reversing Strings" ^ p^ "StringReverser should" ^ "reverse a String" ! e1 ^ "leave an empty String unaffected" ! e2 ^ "not fall down snivelling when someone feeds it null" ! e3 ^ p^ "OptionStringReverser should" ^ "reverse Option[String]" ! e4 ^ "reverse None" ! e5 ^ "Damn it, will someone please fix the universe?" ! e6

def e1 = StringReverser("hello") must_== "olleh" def e2 = StringReverser("") must beEmpty def e3 = StringReverser(null) must beNull def e4 = OptionStringReverser(Some("hello")) must beSome("olleh") def e5 = OptionStringReverser(None) must beNone def e6 = OptionStringReverser(Some(null)) must beSome(null)}

Acceptance specification syntax

def is kicks it off -

class TrivialAcceptanceSpec extends Specification { def is =

^ glues things together. p is a FormattingFragment.

"This is a specification to check reversing Strings" ^ p^ "StringReverser should" ^

"description" ! body creates an Example where body is a method that returns a Result.

"reverse a String" ! e1

where

def e1 = StringReverser("hello") must_== "olleh"

returns MatchResult[Any]Formatting

The specs2 user guide contains a detailed Layout section explaining how acceptance specifications are formatted.

Q: Will anything aid you with keeping acceptance specification syntax neatly lined up?

A: Alas, not yet.

Keep an eye on the mailing list and Scalariform - someone will surely do something soon.Running an acceptance spec

Fire up sbt and run

> test-only prasinous.acceptance.TrivialAcceptanceSpec

[info] == prasinous.acceptance.TrivialAcceptanceSpec ==[info] This is a specification to check reversing Strings[info][info] StringReverser should[info] + reverse a String[info] + leave an empty String unaffected[info] + not fall down snivelling when someone feeds it null[info][info] OptionStringReverser should[info] + reverse Option[String][info] + reverse None[info] + Damn it, will someone please fix the universe?[info][info] Total for specification TrivialAcceptanceSpec[info] Finished in 228 ms[info] 6 examples, 0 failure, 0 error[info]

Acceptance specs are functional by default

package prasinous.acceptance

import org.specs2._

class FunctionalDemoSpec extends Specification { def is =

"My functional spec demo should" ^ "show that only the last Result is returned" ! e1 ^ "fail as expected when results are chained together" ! e2

def e1 = { 1 must beGreaterThan(9999) // this MatchResult is discarded 1 must beLessThanOrEqualTo(1) }

// this fails as expected def e2 = 1 must beGreaterThan(9999) and beLessThanOrEqualTo(1)

}

Yields:

[info] == prasinous.acceptance.FunctionalDemoSpec ==[info] My functional spec demo should[info] + show that only the last Result is returned[error] x fail as expected when results are chained together[error] 1 is less than 9999 (FunctionalDemoSpec.scala:10)[info]

Mutable specs are not

class NotFunctionalDemoSpec extends org.specs2.mutable.Specification { "Mutable specs" should { "fail on any bad expectation" in { e1 // fails here e2 // this is bad too, but we never get here until we fix e1 } "fail on chained bad expectations too" in { e3 } }

def e1 = { 1 must beGreaterThan(9999) // this MatchResult is NOT discarded 1 must beLessThanOrEqualTo(1) } def e2 = { 1 must beGreaterThan(8888) // this would fail but we never get here } def e3 = 1 must beGreaterThan(9999) and beLessThanOrEqualTo(1)}

Yields:

[info] == prasinous.unit.NotFunctionalDemoSpec ==[info] Mutable specs should[error] x fail on any bad expectation[error] 1 is less than 9999 (NotFunctionalDemoSpec.scala:8)[error] x fail on chained bad expectations too[error] 1 is less than 9999 (NotFunctionalDemoSpec.scala:12)

Thrown expectations: a big difference

Eric: If you mix ThrownExpectations to an Acceptance Spec it will change the behavior so that any matcher failing will stop the execution of an example.

Acceptance specs matcher behaviour is to return an Expectable which handles applying the matcher and returning a Result.

Unit specs throw expectations as soon as they fail.

If you want acceptance specs to throw expectations like unit specs, mix in ThrownExpectations.

It doesn't change the functional behaviour of acceptance specs (i.e. only the returned ``Result`` will affect the final outcome), but it's useful for interfacing with other test frameworks.

My spec exposed a bug which was promptly fixed in specs2 1.4, which has just been released.Acceptance spec with thrown expectations mixed in

class ThrownExceptionsDemo extends Specification with ThrownExpectations { def is =

"My functional spec demo should" ^ "show that only the last Result is returned" ! e1 ^ "fail as expected when results are chained together" ! e2

def e1 = { 1 must beGreaterThan(9999) // fails because ThrownExpectations was mixed in 1 must beLessThanOrEqualTo(1) }

// fails because first assumption in chain is bad def e2 = 1 must beGreaterThan(9999) and beLessThanOrEqualTo(1)}

Fails just like a mutable spec because ThrownExpectations is mixed in:

[info] == prasinous.acceptance.ThrownExceptionsDemo ==[info] My functional spec demo should[error] x show that only the last Result is returned[error] 1 is less than 9999 (ThrownExceptionsDemo.scala:9)[error] x fail as expected when results are chained together[error] 1 is less than 9999 (ThrownExceptionsDemo.scala:10)

Migrating to specs2

Eric: The best way to introduce concurrency into examples is to isolate the mutable variables.

NEW Eric posted about specs2 migration on his blog.

unit specs - the path of least resistance acceptance specs - requires complete restructuring, but in exchange for substantial benefits

Migrating to specs2Use case: migrating my own project

On 9 March, I migrated Salat from specs 1.6.7 to specs2 1.0.1. It took about two hours, and it was easy.

My strategy was as follows:

update the specs2 dependencies switch my base testing trait, SalatSpec from using org.specs.Specification to using org.specs2.mutable.Specification address minor syntax changes to the matcher syntax handle cases where I needed to do something before and/or after a unit test

Updating the dependencies

Drop in place and go:

val specs2 = "org.specs2" %% "specs2" % "1.4" % "test"val scalaz = "org.specs2" %% "specs2-scalaz-core" % "6.0.RC2" % "test"

def specs2Framework = new TestFramework("org.specs2.runner.SpecsFramework")override def testFrameworks = super.testFrameworks ++ Seq(specs2Framework)

val snapshots = "snapshots" at "http://scala-tools.org/repo-snapshots"val releases = "releases" at "http://scala-tools.org/repo-releases"

Migrating to specs2: restructuring my test traitBefore

trait SalatSpec extends Specification with PendingUntilFixed with Logging { val SalatSpecDb = "test_salat" detailedDiffs() doBeforeSpec { com.mongodb.casbah.commons.conversions.scala.RegisterConversionHelpers() com.mongodb.casbah.commons.conversions.scala.RegisterJodaTimeConversionHelpers() } doAfterSpec { MongoConnection().dropDatabase(SalatSpecDb) }

}

Migrating to specs2: restructuring my test traitThe preferred way to do this should be using map as shown in Generic specification with setup and teardown steps.After (the way I originally did it)

trait SalatSpec extends Specification with Logging { val SalatSpecDb = "test_salat" override def is = Step { com.mongodb.casbah.commons.conversions.scala.RegisterConversionHelpers() com.mongodb.casbah.commons.conversions.scala.RegisterJodaTimeConversionHelpers() } ^ super.is ^ Step { MongoConnection().dropDatabase(SalatSpecDb) }

}

After (now using map)

trait SalatSpec extends Specification with Logging { override def map(fs: =>Fragments) = Step { com.mongodb.casbah.commons.conversions.scala.RegisterConversionHelpers() com.mongodb.casbah.commons.conversions.scala.RegisterJodaTimeConversionHelpers() } ^ fs ^ Step { MongoConnection().dropDatabase(SalatSpecDb) }}

Migrating to specs2: restructuring my test trait

No more detailedDiffs() - the default settings were good enough: No more PendingUntilFixed - in specs2 this is now part of the common specification features

Setup and teardown (before using map)

doBeforeSpec has been replaced by overriding is with a Step to register Casbah's conversion helpers doAfterSpec has been replaced by using ^ to glue a final step onto the supertrait's is method

Setup and teardown (using map)doBeforeSpec and doAfterSpec have been replaced by using map to clearly define that a Step occurs before my Fragments and after.Migrating to specs2: changes to matchers

The following matcher forms are now preferred:

a must matcher(b)a must not matcher(c)

Before

"My test string" must notContain("bingo")dbo must notHaveKey("aa")

After

"My test string" must not contain("bingo")dbo must not have key("aa")

Migrating to specs2: making things run sequentially

For the most part, the specs in Salat can run concurrently.

However, some examples for SalatDAO required sequential access to shared mutable state in a single MongoDB collection.

class SalatDAOSpec extends SalatSpec {

// which most specs can execute concurrently, this particular spec needs to execute sequentially // to avoid mutating shared state: namely, the MongoDB collection referenced by the AlphaDAO

override def is = args(sequential = true) ^ super.is

You can use just sequential as a shortcut for args(sequential = true) because it's frequently used.Migrating to specs2: using scopes to set up data

Unit specs have Scope, a simple way of creating a new scope with variables that can be re-used in any example.

I used it to isolate the tedium of data setup so that my examples could focus on what I was really trying to achieve.

trait xiScope extends Scope { log.debug("before: dropping %s", XiDAO.collection.getFullName()) XiDAO.collection.drop() XiDAO.collection.count must_== 0L

val xi1 = Xi(x = "x1", y = Some("y1")) val xi2 = Xi(x = "x2", y = Some("y2")) val xi3 = Xi(x = "x3", y = Some("y3")) val xi4 = Xi(x = "x4", y = Some("y4")) val xi5 = Xi(x = "x5", y = None) val _ids = XiDAO.insert(xi1, xi2, xi3, xi4, xi5) _ids must contain(Option(xi1.id), Option(xi2.id), Option(xi3.id), Option(xi4.id), Option(xi5.id)) XiDAO.collection.count must_== 5L}

Migrating to specs2: using a scope

My new xiScope can be used as easily as:

"support using a projection on an Option field to filter out Nones" in new xiScope { // a projection on a findOne that matches xi1 XiDAO.primitiveProjection[String](MongoDBObject("x" -> "x1"), "y") must beSome("y1") // a projection on a findOne that brings nothing back XiDAO.primitiveProjection[String](MongoDBObject("x" -> "x99"), "y") must beNone

val projList = XiDAO.primitiveProjections[String](MongoDBObject(), "y") projList must haveSize(4) projList must contain("y1", "y2", "y3", "y4") // xi5 has a null value for y, not in the list}

Control execution and reporting

Use arguments. It's that easy.

Inside a spec, pass them in to is. For instance, let's say you are working inside a web framework and you want to filter stacktraces to show only your own code.

def is = args(traceFilter = includeTrace("com.foo.confabulator"))

In sbt, you can pass in arguments: the example shown below will output to both console and html.

> test-only com.foo.confabulator.test.TryHarderSpec -- html console

JUnit Integration

class WithJUnitSpec extends SpecificationWithJUnit { "My spec" should { "run in JUnit too" in { success } }}

For IDE support, you can still use @Runner:

import org.junit.runner._import runner._

@RunWith(classOf[JUnitRunner])class WithJUnitSpec extends Specification { "My spec" should { "run in JUnit too" in { success } }}

Matchers: making expectations easy

In specs2, you can define expectations on anything that returns a Result.

Boolean

Standard Results success, failure, anError, pending, etc.

Matcher result - specs2 has built in support for all these and more: Any Option / Either Strings and Numbers Exceptions Iterable and Maps XML and JSON Scalaz Parser Combinator matchers ScalaCheck property Mock expectation DataTable Forms

Where to find information about matchers

Matchers Guide specs2 matcher code specs2 matcher specs

Iterable matchersspecs 1.x:

val list = List(1, 2, 3)list must have size(3)list must containInOrder(1, 2, 3)

specs2

Using only and inOrder we can state this in one shot:

List(1, 2, 3) must contain(1, 2, 3).only.inOrder

JSON matchersThe JSON matchers rely on the standard Scala libs.

/(value) looks for a value at the root of an Array

"""["name", "Joe" ]""" must /("name")

/(key -> value) looks for a pair at the root of a Map

"""{ "name": "Joe" }""" must /("name" -> "Joe")"""{ "name": "Joe" }""" must not /("name2" -> "Joe")

*/(value) looks for a value present anywhere in a document, either as an entry in an Array or as the value for a key in a Map

*/(key -> value) looks for a pair anywhere in the document

JSON matchers can be chained:

"""{ "person": { "name": "Joe" } }""" must /("person") /("name" -> "Joe")

See JsonMatchersSpec in the specs2 project specs for more details.specs2 matchers in the wild

Eric Torreborre's fork of lift/framework is using the new ParserMatchers trait to specify the parsers helpers CombParserHelpersSpec using specs2 compare with original CombParserHelpersSpec using specs 1.x

Casbah is now using specs2 shiny new custom DBObject matchers in DBObjectMatchers usage examples in DSLCoreOperatorsSpec and BarewordOperatorsSpec

TDD on Android using Robolectric with Specs2 https://github.com/jbrechtel/robospecs Salat-Avro has more examples of JSON matchers

Contexts: making things happen when you need themThe specs2 user guide has been updated to provide increased coverage on how to use Contexts - go see that immediately.Also, see this helpful gist showing how to use implicit contexts to reduce duplication.

When you want to make sure that something happens for every example, use any combination of these traits:

Before After Around Outside

Contexts provide an apply method which can be applied to the body of an example so that your code is executed when you need it relative to the example code.

Contexts of the same type can be composed and/or sequenced. Contexts can also extend each other to provide more specific setups.

Context now extends Scope. Whereas Scope brings what's inside into context with a mutable spec, a Context is about having the appropriate method being executed when and where you need it (http connection, database operations).Demonstration of Around (unit spec)

Courtesy of a gist from Eric Torreborre. >> appends an Example to the unit spec's list of fragments.

class UnitSpec extends org.specs2.mutable.Specification {

"This specification has examples which must be executed inside an http session" >> { "Example 1 is executed inside the session" >> http { success } "Example 2 is also executed inside the session" >> http { success } }

object http extends Around { def around[T <% Result](t: =>T) = openHttpSession("test") { t // execute t inside a http session } }}

Eric also has a gist sketching out how to use Outside and AroundOutside.This old spec: specs 1.x

There's always something special in a dusty corner... Now it sees the light of day.

class BadIdea extends Specification { var badIdea = MSet.empty[Int] "My badly set up spec" should { shareVariables() // now we're in for it... doFirst { println("setUp(): hey, who got rid of JUnit?") badIdea = MHashSet.empty[Int] ++= (1 to 100).toSet badIdea must notBeEmpty } "make use of shared mutable state" in { badIdea ++= (101 to 200).toSet badIdea must have size (200) } "have expectations based on previous sequential manipulation of shared mutable state" in { badIdea = badIdea.filter(_ % 2 == 0) badIdea must have size (100) } // MOAR... doLast { println("tearDown(): success!") badIdea.clear() } }}

This old spec: hauled into specs2 with complete violation of intent

import scala.collection.mutable.{Set => MSet, HashSet => MHashSet}

class BadIdea extends org.specs2.mutable.Specification { var badIdea = MSet.empty[Int]

override def is = args(sequential=true)^ Step { println("Doing something beforehand!") badIdea = MHashSet.empty[Int] ++= (1 to 100).toSet badIdea must not be empty } ^ super.is ^ Step { println("Doing something afterwards!") badIdea.clear() }

"My badly set up spec" should { "make use of shared mutable state" in { badIdea ++= (101 to 200).toSet badIdea must have size(200) } "have expectations based on previous sequential manipulation of shared mutable state" in { badIdea = badIdea.filter(_ % 2 == 0) badIdea must have size(100) } // Send help... }}

This old spec: renovated

The simplest way to equip your examples with the state they expect: use Scope to create a reusable state sandbox and rebase the existing expectations to work with the new setup.

class BetterIdea extends Specification {

// happens BEFORE each use case trait testData extends Scope { println("Setting up data JUST for you and your little dog, my pretty.") val betterIdea = (1 to 100).toSet betterIdea must have size(100) }

"My better spec using contexts" should { "force each use case to have its own immutable data" in new testData { val newSet = betterIdea ++ (101 to 200).toSet newSet must have size(200) } "get rid of expectations that depend on shared state" in new testData { val anotherNewSet = betterIdea.filter(_ % 2 == 0) anotherNewSet must have size(50) // changed expectation no longer depends on shared state! } }}

This old spec: what about context instead of scope?

Now, imagine instead of a set, we were using shared mutable state like a database table.

Since I didn't want to introduce deps into my test code, pretend Before is actually populating a database table with known test data. Now using a Context instead of a Spec makes sense.

object setupData extends Before { var betterIdea = Set.empty[Int] def before { betterIdea ++= (1 to 100).toSet betterIdea must have size(100) }}

"or do it with a context instead" >> { "now my example will be executed inside the setupData context" >> setupData { // sub in database access for "setupData.betterIdea" val newSet = setupData.betterIdea ++ (101 to 200).toSet newSet must have size (200) }}

Given - When - Then

In specs 1.x, given, when and then were methods with no added value but displaying these words with a description.

In specs2, Given, When and Then are now RegexSteps that can extract typed values from parameterized text:

class GivenWhenThenSpec extends org.specs2.Specification { def is = "A given-when-then example for GCD" ^ "Given the following number: ${4}" ^ number1 ^ "And the following number: ${2}" ^ number2 ^ "Then the greatest common denominator is ${2}" ^ result ^ end object number1 extends Given[Long] { def extract(text: String): Long = extract1(text).toLong } object number2 extends When[Long, BinaryGCD] { def extract(number1: Long, text: String) = BinaryGCD(number1, extract1(text).toLong) } object result extends Then[BinaryGCD] { def extract(b: BinaryGCD, text: String): Result = b.gcd must_== extract1(text).toLong } case class BinaryGCD(u: Long, v: Long) { def gcd: Long = { /* Some impl */ } }}

Using ScalaCheck to reduce drudgery

For all the ceremony in the previous GivenWhenThenSpec, it's dreadful:

it tests only a single case we have to supply all the inputs and expectations by hand. Yawn. how long would it take us to cover a halfway decent set of data?

Generating test case data by hand is a drag. ScalaCheck to the rescue!Key concepts

a property is a testable unit that specifies the behaviour of a method properties created from functions require an implciit Arbitrary[T] instance generators are responsible for generating data in ScalaCheck

Consult the ScalaCheck User Guide for a detailed explanation of property and generator types.specs2 support for ScalaCheck

specs2 comes with built-in support for ScalaCheck.

extend org.specs2.Specification with org.specs2.ScalaCheck the ScalaCheck trait provides a check function that transforms a function to a org.scalacheck.Prop and then to a Result import step classes (Given, When, etc) from org.specs2.specification.gen instead of org.specs2.specification Given and When steps now return ScalaCheck generators the extract method on Then takes an implicit Arbitrary[T]

More examples

See JsonSpec for an example of using ScalaCheck to generate JSON.Using ScalaCheck

class ScalaCheckGwtSpec extends Specification with ScalaCheck { def is = "Testing Binary GCD calculator" ^ "Given the following number n1" ^ number1 ^ "And the following number n2" ^ number2 ^ "When we take the greatest common denominator" ^ gcd ^ "Then the binary GCD matches the Euclidian GCS" ^ result ^ end object number1 extends Given[Long] { def extract(text: String) = choose(-10L, 10L) } object number2 extends When[Long, (Long, Long)] { def extract(number1: Long, text: String) = for { n2 <- choose(-10L, 10L) } yield (number1, n2) } object gcd extends When[(Long, Long), BinaryGCD] { def extract(numbers: (Long, Long), text: String) = BinaryGCD(numbers._1, numbers._2) } object result extends Then[BinaryGCD] { def extract(text: String)(implicit op: Arbitrary[BinaryGCD]) = { check { (op: BinaryGCD) => op.gcd must_== EuclidianGCD(op.u, op.v) } } }}

Running ScalaCheck

Using all the permutations of data generated by choose, we make the anemic GivenWhenThenSpec far more robust by running our expectation 100 times and checking our Binary GCD algorithm against the results of an indepdendent Euclidian GCD algorithm.

[info] == prasinous.acceptance.ScalaCheckGwtSpec ==[info] Testing Binary GCD calculator[info] Given the following number n1[info] And the following number n2[info] When we take the greatest common denominator[info] + Then the binary GCD matches the Euclidian GCS[info][info] Total for specification ScalaCheckGwtSpec[info] Finished in 59 ms[info] 1 example, 100 expectations, 0 failure, 0 error[info][info] == prasinous.acceptance.ScalaCheckGwtSpec ==

And if we decide to run the expectation 10,000 times, all we have to do is change some inputs to choose.Could you use Given - When - Then with unit specs?

Eric: I'm not actually sure how feasible this is actually because the GivenWhenThen steps in an Acceptance spec are very typechecked. In a mutable spec I would need to do runtime checks and have additional variables.

If there were suport for it, Eric suggests it might look like this:

class ScalaCheckGwtUnitSpec extends Specification with ScalaCheck { "Testing Binary GCD calculator" { "Given the following number n1" ! given { choose(-10L, 10L) } "And the following number n2" ! when { (number1: Long, text: String) => for {n2 <- choose(-10L, 10L)} yield (number1, n2) } "When we take the greatest common denominator" ! when { (numbers: (Long, Long), text: String) => BinaryGCD(numbers._1, numbers._2) } "Then the binary GCD matches the Euclidian GCS " ! then check { (op: BinaryGCD) => op.gcd must_== EuclidianGCD(op.u, op.v) } }}

User Q & AQuestion

Can specs2 be used with ScalaTest? and for what purpose?Answer

Eric:

The specs2 matchers should be usable in ScalaTest (and reciprocally). You could use specs2 Json matchers for example while keeping the ScalaTest reporters and infrastructure.

Possibly the ScalaCheck API or the DataTables as well but I haven't tried it.

Those objects traits can also be reused in JUnit and TestNG modulo some adapation of the ThrownExpectations trait. See for example what is done here: JUnitMatchers.scalaQuestion

Can use cases written by a business user be parsed into a sample acceptance spec with default methods e1, e2, etc. populated?AnswerEric:

Possibly, yes. However my view on the subject is that business users can not conveniently do this. Note that the big difficulty is not so much in writing the first spec but more in maintaining the whole lot.

The long-term answer is that I would like to write a GUI client to support this case: with version control integration, iterative development,...

Actually ThoughtWorks is selling a tool that makes most of this:http://www.thoughtworks-studios.com/agile-test-automationQuestion

Did Given - When - Then come over from Ruby/Cuke?AnswerEric:

Absolutely. My take on it is:* you don't have to use the given-when-then keywords but you can use whatever text you feel is natural* the G-W-T sequence is statically typed: well as much as possible, it is still possible to fail when extracting values from the textQuestion

What else can Given - When - Then be used for?Answer

Eric:

I just see it as a guideline to write clear specifications along the "Arrange-act-assert" paradigm:http://c2.com/cgi/wiki?ArrangeActAssertQuestion

Is it possible to define the body inline when using "description" ! body format?Answer

Eric: Yes.

class InlineFunctionalDemoSpec extends Specification { def is =

"My functional spec demo should" ^ "show that only the last Result is returned" ! { 1 must beGreaterThan(9999) // this MatchResult is discarded 1 must beLessThanOrEqualTo(1) } ^ "fail as expected when results are chained together" ! { 1 must beGreaterThan(9999) and beLessThanOrEqualTo(1) }

}

Question

What is AutoExample for?Answer

Eric:

To avoid repetition between the example description and the example code when the code says it all. For example I used that a lot to specify matchers:

https://github.com/etorreborre/specs2/blob/1.3/src/test/scala/org/specs2/matcher/AnyMatchersSpec.scala

It's also used to enable the "backtick" notation: http://etorreborre.github.com/specs2/guide/org.specs2.guide.SpecStructure.html#Acceptance+specification (see "you can even push this idea further by writing:").Question

What's the rationale behind acceptance specs? Isn't the way unit specs mingle text and code a benefit?Answer

Eric:

My own rationale is the following. By being able to read the whole spec text with one glance I can better think about what I expect from my system. So when I implement something new, I usually spend some time adding the examples which make sense before implementing them. I also found my specs easier to understand when revisiting them after a few weeks.Discussion

What is the actual process of putting together an acceptance spec?Writing process

You might begin by just stubbing out expectations with inline results:

class AcceptanceProcessDraft extends Specification { def is = "first example" ! pending ^ "second example" ! pending}

Once the examples are clear, extract the results out into methods and begin populating them:

class AcceptanceProcessSecondDraft extends Specification { def is = "first example" ! e1 ^ "second example" ! e2 ^ "third example" ! e3

def e1 = pending def e2 = pending def e3 = pending}

Discussion

What is the actual process of putting together an acceptance spec?Notes from Eric

There have been numerous discussion of acceptance spec style and purpose on the specs2 mailing list.

Eric:

One thing I noticed is that I often split examples in two:"this should do that" ! e1^becomes

"this should do that" ^ "in this case" ! e1^ "in that case" ! e2^

Discussion

Compare and contrast specs/specs2 with ScalaTest.My personal experience

Speaking personally, I started out with ScalaTest at work but quickly found specs to be a better tool for me.

Although ScalaTest appeared more direct to me at first, coming from a JUnit/TestNG background, over a period of two months I became dissatisfied:

the errors when tests failed were difficult to work with. When I was just starting out in Scala, trying to figure out where in an N-deep level nest of anonymous inner classes some expectation failed was very daunting to me. ScalaTest dynamic matchers seemed initially appealing when I wanted to check values within an object graph but turned into a maintenance nightmare when I refactored model objects.

I saw that some open source projects I liked were using specs, so I gave it a try. I appreciated the clean syntax of specs, and I liked the matcher syntax more than ScalaTest's.

In simple testing setups ScalaTest and specs appear quite similar, so my transition from ScalaTest to specs was quite rapid. When my test cases failed, the errors made it extremely easy to target the failing lines.

Discussion

Compare and contrast specs/specs2 with ScalaTest.My personal experience, continued

Although I came to specs for its simplicity, in the long run what kept me using specs was its power. So it was a natural for me to migrate Salat to specs2 when it was released. specs2, although a complete rewrite, had everything I liked about specs 1.x plus many new powerful features I could use to isolate mutability in my specs.

That said, Akka uses ScalaTest, and I quite admire the style and thoroughness of their tests as a model for anyone who wants to use ScalaTest.

There have been numerous discussions online. Here's a good StackOverflow thread with a comparison from Bill Venners:

What's the difference between ScalaTest and Scala Specs unit test frameworks?

Online resources

http://specs2.org Quick Start Updated User Guide Sample project User Group @specs2.org

Eric Torreborre @etorreborre Blog NEW specs2 migration guide

Other projects mentioned in these slides Scalaz Scalacheck Mockito Fitnesse

Sample code

Official specs2 examples My sample code for this specs2 presentation

Salat uses specs2 Main conversion changeset Cleanup work SalatDAOSpec showing using contexts and demonstrating how to run a test sequentially SalatSpec - a mutable Specification trait that has before and after steps

Useful things to think about

specs2 Philosophy Josh Suereth's NEScala symposium talk, Implicits without the import tax

Thank you

@etorreborre for writing and beautifully documenting specs2, and so much more: for assisting at every stage in the preparation and remix of this presentation for clarifying the user guide in response to some of the questions from this presentation for being so responsive at more hours of the day and night than I could possibly expect (especially given the time difference between New York and Sydney - it's clear he's someone who cares a lot about helping people to understand and use specs2) @softprops for picture-show Novus Partners for hosting this ny-scala meetup @n8han for filming this talk all of the attendees at the presentation for displaying such interest and asking so many questions! everyone who has re-tweeted this presentation, for helping to create a vital and interesting discussion around BDD testing frameworks in Scala

package knoldus.Specs2import org.specs2.mutable.Specification

import org.specs2.mutable

class HelloWorldSpec extends Specification {

"The 'Hello world' string" should { "contain 11 characters" in { "Hello world" must have size(11) } "start with 'Hello'" in { "Hello world" must startWith("Hello") } "end with 'world'" in { "Hello world" must endWith("world") }}}

Guide to write Acceptance Specifications

Guide to write Acceptance Specifications

Extend the org.specs2.Specification trait

are functional when extending the default org.specs2.Specification trait

Must define a method called is that takes a Fragments object, which is composed of an optional SpecStart , a list of Fragment objects an options SpecEnd

Extend the org.specs2.Specification trait

are functional when extending the default org.specs2.Specification trait

Must define a method called is that takes a Fragments object, which is composed of an optional SpecStart , a list of Fragment objects an options SpecEnd

Creating Acceptance SpecificationsCreating Acceptance Specifications

package knoldus.Specs2import org.specs2._

class HelloWorldAcceptanceSpec extends Specification { def is =

"This is a specification to check the 'Hello world' string" ^ p ^ "The 'Hello world' string should" ^ "contain 11 characters" ! e1 ^ "start with 'Hello'" ! e2 ^ "end with 'world'" ! e3 ^ end

def e1 = "Hello world" must have size (11) def e2 = "Hello world" must startWith("Hello") def e3 = "Hello world" must endWith("world")}

package knoldus.Specs2import org.specs2._

class HelloWorldAcceptanceSpec extends Specification { def is =

"This is a specification to check the 'Hello world' string" ^ p ^ "The 'Hello world' string should" ^ "contain 11 characters" ! e1 ^ "start with 'Hello'" ! e2 ^ "end with 'world'" ! e3 ^ end

def e1 = "Hello world" must have size (11) def e2 = "Hello world" must startWith("Hello") def e3 = "Hello world" must endWith("world")}

Acceptance Specifications are functional

Acceptance Specifications are functional

The default Specification trait in specs2 is functional: the Result of an example is always given by the last statement of its body. This example will never fail because the first expectation is "lost":

"my example on strings" ! e1 // will never fail!

def e1 = { "hello" must have size(10000) // because this expectation will not be returned,... "hello" must startWith("hell") }

So the correct way of writing the example is:

"my example on strings" ! e1 // will fail

def e1 = "hello" must have size(10000) and startWith("hell")

The default Specification trait in specs2 is functional: the Result of an example is always given by the last statement of its body. This example will never fail because the first expectation is "lost":

"my example on strings" ! e1 // will never fail!

def e1 = { "hello" must have size(10000) // because this expectation will not be returned,... "hello" must startWith("hell") }

So the correct way of writing the example is:

"my example on strings" ! e1 // will fail

def e1 = "hello" must have size(10000) and startWith("hell")

MatchersMatchers

there are many ways to define expectations in specs2. You can define expectations with anything that returns a Result:

Boolean Standard result Matcher result Scalacheck property Mock expectation DataTable Forms

there are many ways to define expectations in specs2. You can define expectations with anything that returns a Result:

Boolean Standard result Matcher result Scalacheck property Mock expectation DataTable Forms

Boolean ResultBoolean Resultthis is the simplest kind of result you can define for an expectation but also the least expressive!

Here's an example:

"This is hopefully true" ! (1 != 2)

This can be useful for simple expectations but a failure will give few information on what went wrong:

"This is hopefully true" ! (2 != 2) // fails with 'the value is false',...

this is the simplest kind of result you can define for an expectation but also the least expressive!

Here's an example:

"This is hopefully true" ! (1 != 2)

This can be useful for simple expectations but a failure will give few information on what went wrong:

"This is hopefully true" ! (2 != 2) // fails with 'the value is false',...

Standard ResultStandard ResultSome standard results can be used when you need specific result meanings:

success: the example is ok failure: there is a non-met expectation anError: a non-expected exception occurred skipped: the example is skipped possibly at runtime because some conditions are not met. A more specific message can be created with Skipped("my message") pending: usually means "not implemented yet", but a specific message can be created with Pending("my message")

Two additional results are also available to track the progress of features:

done: a Success with the message "DONE" todo: a Pending with the message "TODO"

Some standard results can be used when you need specific result meanings:

success: the example is ok failure: there is a non-met expectation anError: a non-expected exception occurred skipped: the example is skipped possibly at runtime because some conditions are not met. A more specific message can be created with Skipped("my message") pending: usually means "not implemented yet", but a specific message can be created with Pending("my message")

Two additional results are also available to track the progress of features:

done: a Success with the message "DONE" todo: a Pending with the message "TODO"

Matcher ResultMatcher Resultthe most common matchers are automatically available when extending the Specification trait:

1 must beEqualTo(1) the normal way1 must be_==(1) with a shorter matcher1 must_== 1 my favorite!1 mustEqual 1 if you dislike underscores1 should_== 1 for should lovers1 === 1 the ultimate shortcut1 must be equalTo(1) with a literate style

the most common matchers are automatically available when extending the Specification trait:

1 must beEqualTo(1) the normal way1 must be_==(1) with a shorter matcher1 must_== 1 my favorite!1 mustEqual 1 if you dislike underscores1 should_== 1 for should lovers1 === 1 the ultimate shortcut1 must be equalTo(1) with a literate style

Iterable MatchersIterable Matchersspecs 1.x:


specs2



specs 1.x:


specs2



JSON MatchersJSON Matchersspecs 1.x:


specs2



specs 1.x:


specs2



JSON MatchersJSON Matchers









MockingMockingimport org.specs2.mock._ class MockitoSpec extends Specification { def is =

"A java list can be mocked" ^ "You can make it return a stubbed value" ! c().stub^ "You can verify that a method was called" ! c().verify^ "You can verify that a method was not called" ! c().verify2^ end case class c() extends Mockito { val m = mock[java.util.List[String]] // a concrete class would be mocked with: mock[new java.util.LinkedList[String]] def stub = { m.get(0) returns "one" // stub a method call with a return value m.get(0) must_== "one" // call the method } def verify = { m.get(0) returns "one" // stub a method call with a return value m.get(0) // call the method there was one(m).get(0) // verify that the call happened } def verify2 = there was no(m).get(0) // verify that the call never happened } }

import org.specs2.mock._ class MockitoSpec extends Specification { def is =

"A java list can be mocked" ^ "You can make it return a stubbed value" ! c().stub^ "You can verify that a method was called" ! c().verify^ "You can verify that a method was not called" ! c().verify2^ end case class c() extends Mockito { val m = mock[java.util.List[String]] // a concrete class would be mocked with: mock[new java.util.LinkedList[String]] def stub = { m.get(0) returns "one" // stub a method call with a return value m.get(0) must_== "one" // call the method } def verify = { m.get(0) returns "one" // stub a method call with a return value m.get(0) // call the method there was one(m).get(0) // verify that the call happened } def verify2 = there was no(m).get(0) // verify that the call never happened } }

FormsForms

Forms are a way to represent domain objects or services, and declare expected values in a tabular format. Forms can be designed as reusable pieces of specification where complex forms can be built out of simple ones.

class SpecificationWithForms extends Specification with Forms { def is =

"The address must be retrieved from the database with the proper street and number" ^ Form("Address"). tr(prop("street", actualStreet(123), "Oxford St")). tr(prop("number", actualNumber(123), 20)) ^ end }

Forms are a way to represent domain objects or services, and declare expected values in a tabular format. Forms can be designed as reusable pieces of specification where complex forms can be built out of simple ones.

class SpecificationWithForms extends Specification with Forms { def is =

"The address must be retrieved from the database with the proper street and number" ^ Form("Address"). tr(prop("street", actualStreet(123), "Oxford St")). tr(prop("number", actualNumber(123), 20)) ^ end }

Running Specification Using JunitRunning Specification Using JunitWith Junit We can run test as this



With Junit We can run test as this



Running Specification Using SBTRunning Specification Using SBTWith Sbt We can run test as this

For console OutPut Add this line in your build.sbttestOptions in Test += Tests.Argument("console")And run test-only classFileName – console

For html outputAdd dependencies"org.pegdown" % "pegdown" % "1.0.2"testOptions in Test += Tests.Argument("html")And run test-only classFileName – html

For html and console outputtestOptions in Test += Tests.Argument("html",console)And run test-only classFileName – html console

With Sbt We can run test as this

For console OutPut Add this line in your build.sbttestOptions in Test += Tests.Argument("console")And run test-only classFileName – console

For html outputAdd dependencies"org.pegdown" % "pegdown" % "1.0.2"testOptions in Test += Tests.Argument("html")And run test-only classFileName – html

For html and console outputtestOptions in Test += Tests.Argument("html",console)And run test-only classFileName – html console

ThanksThanks

specs2

Technology

hello world string

specification traitmust

example isalways

val list

return value

specs2import org

java list

specificationimport