make your testing groovy - events.static.linuxfound.org · what is groovy? “groovy is like a...

© A

SE

RT

2006-2

016

Make your testing Groovy

Dr Paul KingGroovy Lead for Object Computing Inc.@paulk_asert

http:/slideshare.net/paulk_asert/make-tests-groovy

https://github.com/paulk-asert/MakeTestingGroovy

© A

SE

RT

2006-2

016

Why test with Groovy?

Why test with Groovy?

• Wrong first question!

• … but great second question?

• Consider first the task at hand and

the organization and people fit!

© A

SE

RT

2006-2

016

“People Fit” / “Organization Fit”

• People– Developers (familiarity with languages)

– Testers (tools language familiarity)

– Responsibility to create/run/maintain

– BAs/SMEs (ability to read technical tests)

– Consider both development and maintenance

– Expected feedback from tests (JUnit/Reports)

• Organization– Maturity level

– Degree of automation

– Tools in use

– Need for adaptability

© A

SE

RT

2006-2

016

Why Test With Groovy?

• Advantages– Easy to learn

– Particularly good for Java shops

– Easy to plug and play different testing tools

– Good community & tools for professional agile

• Disadvantages– Requires JVM

– Less advantages if your developers using

Python, .Net, PHP

– Maybe your testers already know Ruby

© A

SE

RT

2006-2

016

Scripting/Dynamic Languages

• Advantages– Lend themselves to succinct code/DSLs

– Powerful

– Increased Refactoring

– Increased Reuse

– Less prone to Brittleness

– Flexibility for tool integration

– Open APIs provide extensibility

• Disadvantages– Can be too low level

– Sometimes less tool support

© A

SE

RT

2006-2

016

Test Characteristics…

• Coverage/Traceability– Requirement coverage/traceability

– Code coverage: line, branch, path, state

– Transactional Tracing

• Purpose– Unit, Integration, System, Customer

• Manageability– Removing duplication

– Managing lifecycle: shared setUp/tearDown

(@Before @After)

© A

SE

RT

2006-2

016

…Test Characteristics

• Handling test data– Data-driven, databases, Spreadsheets, tables,

keywords, random, model-driven

– Large Test Volumes

– Speed of feedback, performance testing

• Tool evolution, longevity, cost,

support, documentation– Open Source/Commercial, Critical

mass/popularity

• Combinability

© A

SE

RT

2006-2

016

Key Testing Practices…

• Use testing DSL’s

• Look to move up the testing stack– It used to be all about the driver

– Now the driver is hidden in the

framework or tool stack

• Apply good testing practices– Pareto analysis, bug clusters, mutation

testing, test early, all pairs/equivalence

partitions/orthogonal array testing, risk-based

test selection, coding for testability, use CI,

boundary value analysis, defensive

programming

© A

SE

RT

2006-2

016

…Key Testing Practices

• Plug and play testing tools– Run different drivers with different runners

and different tools

• Complement automated tests with

exploration

• Expand testing scope– Test environment readiness, test

deployments

© A

SE

RT

2006-2

016

Groovy and Testing Tool Spectrum*

© A

SE

RT

2006-2

016

* Tools/libraries/frameworks don't always neatly fall into one category – still useful conceptually

© A

SE

RT

2006-2

016

What is Groovy?

“Groovy is like a super version of Java.

It leverages Java features but

adds productivity features and

provides great flexibility and

extensibility.”

Groovy = Java – boiler plate code+ closures (1st class FP)+ extensible type system+ runtime & compile-timemetaprogramming

+ flexible grammar (DSLs)+ scripting & GDK library

Groovy starter

© A

SE

RT

2006-2

016

System.out.println("Hello, World!"); // supports Java syntax

println 'Hello, World!' // but can remove some syntax

String name = 'Guillaume' // Explicit typing/awareness

println "$name, I'll get the car." // Gstring (interpolation)

def longer = """${name}, the car

is in the next row.""" // multi-line, implicit type

assert 0.5 == 1/2 // BigDecimal equals()

assert 0.1 + 0.2 == 0.3 // and arithmetic

def printSize(obj) { // implicit/duck typing

print obj?.size() // safe dereferencing

}

def pets = ['ant', 'bee', 'cat'] // literal list syntax

pets.each { pet -> // closure support

assert pet < 'dog' // overloading '<' on String

} // or: for (pet in pets)...

Target audience

• Testing for developers– Organising tests

– Running tests

– Data generation

– Mocking

– What can be tested

– Code coverage

• Testing beyond developers– Structuring

– Better reporting

– Data-driven

– Domain specific language (DSL)

– Test coverage

Testing Frameworks

• None

• JUnit 3

• JUnit 4

• JUnit 5

• TestNG

• Spock

Testing Frameworks

• None ???

• JUnit 3

• JUnit 4

• JUnit 5

• TestNG

• Spock

Testing Frameworks

• None ???– Groovy deems testing so important

that it comes with built-in testing:

– Built-in asserts, mocks

– Built-in JUnit 3 GroovyTestCase and utilities

– Built-in runners for tests

– Bundled JUnit 4

• JUnit 3

• JUnit 4

• JUnit 5

• TestNG

• Spock

Built-in assertions

© A

SE

RT

2006-2

016

import static Converter.celsius

assert 20 == celsius(68)assert 35 == celsius(95)assert -17 == celsius(0).toInteger()assert 0 == celsius(32)

class Converter {static celsius (fahrenheit) {

(fahrenheit - 32) * 5 / 9}

}

Built-in assertions

• But what about errors?

• Groovy’s power Assert mechanism gives

a friendly description of what went wrong

© A

SE

RT

2006-2

016

Built-in assertions

• But what about errors?

• Groovy’s power Assert mechanism gives

a friendly description of what went wrong

© A

SE

RT

2006-2

016

GroovyTestCase (extends JUnit 3)

© A

SE

RT

2006-2

016

import org.junit.Assertimport static Converter.celsiusimport static org.junit.Assert.assertEquals

class SimpleGroovyTest extends GroovyTestCase {void testShouldConvert() {assert Converter.celsius(68) == 20assert Converter.celsius(212) == 100, "Should convert boiling"assertEquals("Should convert freezing", 0.0, celsius(32.0))assertEquals("Should convert nearly freezing", 0.0, celsius(32.1), 0.1)

}}

JUnit4

© A

SE

RT

2006-2

016

import org.junit.Testimport static org.junit.Assert.assertEqualsimport static Converter.celsius

class SimpleJUnit4Test {@Testvoid shouldConvert() {assert celsius(68) == 20assert celsius(212) == 100, "Should convert boiling"assertEquals("Should convert freezing", 0.0, celsius(32.0))assertEquals("Also for nearly freezing", 0.0, celsius(32.1), 0.1)

}}

JUnit5

© A

SE

RT

2006-2

016

@Grab('org.junit.platform:junit-platform-runner:1.0.0-M2')@Grab('org.junit.jupiter:junit-jupiter-engine:5.0.0-M2')import org.junit.jupiter.api.Testimport org.junit.platform.runner.JUnitPlatformimport org.junit.runner.RunWithimport static Converter.celsius

@RunWith(JUnitPlatform)class ConverterJUnit5Tests {

@Testvoid freezing() {

assert celsius(32) == 0}@Testvoid boiling() {

assert celsius(212) == 100}

}

JUnit5

© A

SE

RT

2006-2

016

@Grab('org.junit.platform:junit-platform-runner:1.0.0-M2')@Grab('org.junit.jupiter:junit-jupiter-engine:5.0.0-M2')import org.junit.jupiter.api.Testimport org.junit.platform.runner.JUnitPlatformimport org.junit.runner.RunWithimport static Converter.celsius

@RunWith(JUnitPlatform)class ConverterJUnit5Tests {

@Testvoid freezing() {

assert celsius(32) == 0}@Testvoid boiling() {

assert celsius(212) == 100}

}

Spock - BDD style

© A

SE

RT

2006-2

016

@Grab('org.spockframework:spock-core:1.0-groovy-2.4')import spock.lang.Specification

class GivenWhenThenSpec extends Specification {

def "test adding a new item to a set"() {given:def items = [4, 6, 3, 2] as Set

when:items << 1

then:items.size() == 5

}}

Parameterized

© A

SE

RT

2006-2

016

import org.junit.Test

import org.junit.runner.RunWith

import org.junit.runners.Parameterized

import org.junit.runners.Parameterized.Parameters

import static Converter.celsius

@RunWith(Parameterized)

class DataDrivenJUnitTest {

private c, f, scenario

@Parameters static scenarios() {[

[0, 32, 'Freezing'],

[20, 68, 'Garden party conditions'],

[35, 95, 'Beach conditions'],

[100, 212, 'Boiling']

]*.toArray()}

DataDrivenJUnitTest(c, f, scenario)

this.c = c

this.f = f

this.scenario = scenario

}

@Test void convert() {

def actual = celsius(f)

def msg = "$scenario: ${f}°F should convert into ${c}°C"

assert c == actual, msg

}

}

Spock

© A

SE

RT

2006-2

016

@Grab('org.spockframework:spock-core:1.0-groovy-2.4')import spock.lang.*import static Converter.celsius

class SpockDataDriven extends Specification {def "test temperature scenarios"() {expect:celsius(tempF) == tempC

where:scenario | tempF || tempC'Freezing' | 32 || 0'Garden party conditions' | 68 || 20'Beach conditions' | 95 || 35'Boiling' | 212 || 100

}}

Spock - Celsius

© A

SE

RT

2006-2

016

@Grab('org.spockframework:spock-core:1.0-groovy-2.4')

import spock.lang.*

import static Converter.celsius

class SpockDataDriven extends Specification {

@Unroll

def "Scenario #scenario: #tempFºF should convert to #tempCºC"() {

expect:

celsius(tempF) == tempC

where:

scenario | tempF || tempC

'Freezing' | 32 || 0

'Garden party conditions' | 68 || 20

'Beach conditions' | 95 || 34

'Boiling' | 212 || 100

}

}

Spock - Celsius

© A

SE

RT

2006-2

016

@Grab('org.spockframework:spock-core:1.0-groovy-2.4')

import spock.lang.*

import static Converter.celsius

class SpockDataDriven extends Specification {

@Unroll

def "Scenario #scenario: #tempFºF should convert to #tempCºC"() {

expect:

celsius(tempF) == tempC

where:

scenario | tempF || tempC

'Freezing' | 32 || 0

'Garden party conditions' | 68 || 20

'Beach conditions' | 95 || 34

'Boiling' | 212 || 100

}

}

Spock - Mocks

© A

SE

RT

2006-2

016

@Grab('org.spockframework:spock-core:1.0-groovy-2.4')import spock.lang.Specification

class SpockMock extends Specification {

def "buy ticket for a movie theater"() {given:def purchase = new Purchase("Lord of the Rings", 2)MovieTheater theater = Mock()theater.hasSeatsAvailable("Lord of the Rings", 2) >> true

when:purchase.fill(theater)

then:purchase.completed1 * theater.purchaseTicket("Lord of the Rings", 2)

}

}

Spock - Mocks

© A

SE

RT

2006-2

016

@Grab('org.spockframework:spock-core:1.0-groovy-2.4')import spock.lang.Specification

class SpockMock extends Specification {

def "buy ticket for a movie theater"() {given:def purchase = new Purchase("Lord of the Rings", 2)MovieTheater theater = Mock()theater.hasSeatsAvailable("Lord of the Rings", 2) >> true

when:purchase.fill(theater)

then:purchase.completed1 * theater.purchaseTicket("Lord of the Rings", 2)

}

}

Spock - Mocks

© A

SE

RT

2006-2

016

@Grab('org.spockframework:spock-core:1.0-groovy-2.4')import spock.lang.Specification

class SpockMockWildcards extends Specification {def "cannot buy a ticket when the movie is sold out"() {given:def purchase = new Purchase("Lord of the rings", 2)MovieTheater theater = Mock()

when:theater.hasSeatsAvailable(_, _) >> falsepurchase.fill(theater)

then:!purchase.completed0 * theater.purchaseTicket(_, _)

}}

Spock - Mocks

© A

SE

RT

2006-2

016

@Grab('org.spockframework:spock-core:1.0-groovy-2.4')import spock.lang.Specification

class SpockMockClosureChecks extends Specification {def "on couples night tickets are sold in pairs"() {given:def purchase = new Purchase("Lord of the Rings", 2)MovieTheater theater = Mock()theater.hasSeatsAvailable("Lord of the Rings", 2) >> true

when:purchase.fill(theater)

then:1 * theater.purchaseTicket(_, { it % 2 == 0 })

}}

Assertion frameworks

• None

• JUnit 3 assertions

• Hamcrest

• FEST

• Google truth

assertThat(googleColors).containsNoneOf(PINK, BLACK, WHITE, ORANGE)

assert ![PINK, BLACK, WHITE, ORANGE].any {

color -> color in googleColors }

Property-based testing

• Agile testing game (TDD)– Minimum test code to steer design of minimal

production code with desired business functionality

but 100% code coverage

– “Grey box” testing

– Rarely used with functional programming© A

SE

RT

2006-2

016

Property-based testing

• Agile testing game (TDD)– Minimum test code to steer design of minimal

production code with desired business functionality

but 100% code coverage

– “Grey box” testing

– Rarely used with functional programming

– Instead validate certain properties

© A

SE

RT

2006-2

016

Property-based testing

© A

SE

RT

2006-2

016

@Grab('net.java.quickcheck:quickcheck:0.6')

import static net.java.quickcheck.generator.PrimitiveGenerators.*

import static java.lang.Math.round

import static Converter.celsius

def gen = integers(-40, 240)

def liquidC = 0..100

def liquidF = 32..212

100.times {

int f = gen.next()

int c = round(celsius(f))

assert c <= f

assert c in liquidC == f in liquidF

}

Property-based testing with Spock

© A

SE

RT

2006-2

016

https://github.com/Bijnagte/spock-genesis

Property-based testing: spock genesis

© A

SE

RT

2006-2

016

@Grab('com.nagternal:spock-genesis:0.6.0')@GrabExclude('org.codehaus.groovy:groovy-all')import spock.genesis.transform.Iterationsimport spock.lang.Specificationimport static Converter.celsiusimport static java.lang.Math.roundimport static spock.genesis.Gen.integer

class ConverterSpec extends Specification {def liquidC = 0..100def liquidF = 32..212

@Iterations(100)def "test phase maintained"() {

given:int tempF = integer(-40..240).iterator().next()

when:int tempC = round(celsius(tempF))

then:tempC <= tempFtempC in liquidC == tempF in liquidF

}…

Property-based testing: spock genesis

© A

SE

RT

2006-2

016

@Grab('com.nagternal:spock-genesis:0.6.0')@GrabExclude('org.codehaus.groovy:groovy-all')import spock.genesis.transform.Iterationsimport spock.lang.Specificationimport static Converter.celsiusimport static java.lang.Math.roundimport static spock.genesis.Gen.integer

class ConverterSpec extends Specification {def liquidC = 0..100def liquidF = 32..212

@Iterations(100)def "test phase maintained"() {

given:int tempF = integer(-40..240).iterator().next()

when:int tempC = round(celsius(tempF))

then:tempC <= tempFtempC in liquidC == tempF in liquidF

}…

…

@Iterations(100)def "test order maintained"() {

given:int tempF1 = integer(-273..999).iterator().next()int tempF2 = integer(-273..999).iterator().next()

when:int tempC1 = round(celsius(tempF1))int tempC2 = round(celsius(tempF2))

then:(tempF1 <=> tempF2) == (tempC1 <=> tempC2)

}}

Case Study

Case Study

Case Study

Case Study

Case Study

Case Study

Case Study

Case Study

Case Study

Case Study

Case Study

Case Study

Case Study

All Combinations

All Combinations Case Study

All Pairs

All Pairs Case Study

GPars

• Library classes and DSL allowing

you to handle tasks concurrently:

– Data Parallelism map, filter, reduce functionality

in parallel with parallel array support

– Asynchronous functions extend the Java

executor services to enable multi-threaded

closure processing

– Dataflow Concurrency supports natural

shared-memory concurrency model, using

single-assignment variables

– Actors provide Erlang/Scala-like actors

including "remote" actors on other machines

– Safe Agents provide a non-blocking mt-safe

reference to mutable state; like "agents" in Clojure

58

Case Study with GPars

Constraint/Logic Programming

• Description– Style of programming where relations between

variables are stated in the form of constraints

– First made popular by logic programming languages

such as Prolog but the style is now also used outside

logic programming specific languages

– Constraints differ from the common primitives of

other programming languages in that they do not

specify one or more steps to execute but rather the

properties of a solution to be found

– Popular libraries used with Groovy supporting

constraint programming include Gecode/J, Choco

and tuProlog

– We'll look at Choco as an example

Case Study with Constraint Programming

• You have been asked to set up some test

cases representing the Simpsons’ weekly

blogging habits

• After some careful study you observe the

following strange behavior– They never blog on the same day

– Marge blogs only on a Saturday or Sunday

– Maggie blogs only on a Tuesday or Thursday

– Lisa blogs only on a Monday, Wednesday or Friday

– Bart blogs only on the day after Lisa

– Homer only blogs if noone else blogged the previous

day and doesn't allow anyone to blog the next day

Case Study with Constraint Programming

Case Study with Constraint Programming

Case Study with Constraint Programming

ModelJUnit

ModelJUnit

ModelJUnit

Case Study with ModelJUnit

Case Study with ModelJUnit

Case Study with ModelJUnit

Case Study with ModelJUnit

Case Study with ModelJUnit

Testing DSLs

• Low-level DSL

• Higher-level DSL

post blog from Bart with title "Bart rulz!" and category School and content "Cowabunga Dude!"

More Information: Groovy in Action