Drupal Testing 7 → 8

DrupalCafe20 november 2014

Hans de Raad

Whoami?

● Open source business owner:– Drupal

– Kolab

– OpenSUSE

● Projectmanagement, security testing, regulatory affairs, classical music, etc

● Happy to be here!

I am not a D8 expert

● This presentation is based on the input from talks from DrupalCon Prague and Amsterdam (and a lot of reading).

● It is also a way for me to explore the new concepts in D8 – And share my enthousiasm about it!

● I have a generic programming (mostly PHP and Java) background, so programming for Drupal has always felt a bit strange (functional → OO).– So glad to see that solved in D8!

History of testing in Drupal

● Since 2007 – Drupal 6– SimpleTest and Coder

● Targeted at local development deployments and qa.drupal.org

● This stimulated a number of best practises– The encouragement of adding tests to contrib modules

– The availability of a testing framework for all developers

Limitations SimpleTest

● First of all, the project isn't very active anymore.● It's not very useful for BDD testing (and AJAX)

– Basically anything else than PHP function testing.

● It doesn't really encourage TDD style unit testing, it is a bit inbetween TDD and BDD.

● There aren't very much integration plugins with other testing solutions.

● All in all, it's an aging solution.

Qunit

● Javascript Unit testing● Used by jQuery for its unit tests● Drawbacks:

– JS functions are normally tested within specific context

– Designed to run in a browser

– It's unit testing in an area (user interface) where BDD seems more appropriate.

Testing methodologies

● Test Driven Development● Behavior Driven Development● Others?

– Acceptance Test Driven Development

Test Driven Development

● Definition:– Test-driven development (TDD) is a software

development process that relies on the repetition of a very short development cycle: first the developer writes an (initially failing) automated test case that defines a desired improvement or new function, then produces the minimum amount of code to pass that test, and finally refactors the new code to acceptable standards.

Test Driven Development

● Add a test● Run all tests and see if the new one fails● Implement (new) feature● Run tests again● Refactor code

if necessary● Repeat process

Behaviour Driven Development

● Definition:– In software engineering, behavior-driven development

(abbreviated BDD) is a software development process based on test-driven development (TDD).Behavior-driven development combines the general techniques and principles of TDD with ideas from domain-driven design and object-oriented analysis and design to provide software developers and business analysts with shared tools and a shared process to collaborate on software development, with the aim of delivering "software that matters".

Behaviour Driven Development

● Behavior-driven development was developed by as a response to the issues encountered in test-driven development:– Where to start in the process?

– What to test and what not to test?

– How much to test in one go?

– What to call the tests?

– How to understand why a test fails?

● Acceptance tests should be written using the standard agile framework of a User story: "As a [role] I want [feature] so that [benefit]". – Acceptance criteria should be written in terms of scenarios and implemented as

classes: Given [initial context], when [event occurs], then [ensure some outcomes].

BDD: Scenario format

● Title (one line describing the story)– Narrative:

● As a [role]● I want [feature]● So that [benefit]

● Acceptance Criteria: (presented as Scenarios)– Scenario 1: Title

● Given [context]– And [some more context]...

– When [event]– Then [outcome]– And [another outcome]...

● Scenario 2: ...

Focus: Acceptance testing

● A sign of a projects maturity is when testing starts to focus on achieving and measuring business objectives.

● That is exactly what is happening now.● No client wants to buy a Drupal framework with X

well configured modules– They want a web based (content management) system

– They want a website.

Focus: Testing business value

● In traditional testing methodologies the primary focus used to be hardcore code tests

● With Agile development methodologies this is changing to incorporate testing for actual business value (functionality)

● This is something you can involve your client with (yay!)!

Tools for testing

● Unit testing● Behaviour testing● User interaction simulation● Visual/optical testing

Tools: PHPUnit

● Next generation Unit testing for PHP– Although it has been around for quite some time.

● Integration with many IDEs/editors● Better integration with CI servers like Jenkins● Plugin architecture● The standard in PHP testing for Symfony, Zend

Framework, etc● Most important: very actively maintained

Tools: PHPUnit how does it work?

class StackTest extends PHPUnit_Framework_TestCase

{

public function testPushAndPop()

{

$stack = array();

$this->assertEquals(0, count($stack));

array_push($stack, 'foo');

$this->assertEquals('foo', $stack[count($stack)-1]);

$this->assertEquals(1, count($stack));

$this->assertEquals('foo', array_pop($stack));

$this->assertEquals(0, count($stack));

}

}

Tools: PhantomAS

● Performance testing– Both rendering (PhantomJS) and JS stats (like

jQuery operations)

● Multi device simulation for responsive sites● Ability to integrate with CI servers like Jenkins● Renders reports from metrics in JSON, csv, or

through Elasticsearch or StatsD and Graphite.

Tools: PhantomAS how does it work?

phantomas : {

grunt : {

options : {

assertions : {

'assetsWithQueryString' : 3, // receive warning, when there are more than 3 assets with a query string

'bodyHTMLSize' : 10500, // receive warning, when the bodyHTMLsize is bigger than 10500

'jsErrors' : 0 // receive warning, when JS errors appear

}

indexPath : './phantomas/',

options : {

'timeout' : 30

},

url : 'http://gruntjs.com/'

}

}

}

Tools: PhantomAS example output

Tools: Behat and mink

● BDD testing● Drupal extension to Behat● Simulates the interaction between application and

browser (user)– It has several drivers:

● Selenium or the more modern variant Sahi● Wunit● Etc

● Module: Drupal Extension (for Behat and Mink)

Tools: Drupal extension to Behat and Mink

From their website:● The Drupal Extension to Behat and Mink assists in the

performance of these common Drupal testing tasks:– Set up test data with Drush or the Drupal API– Define theme regions and test data appears within them– Clear the cache, log out, and other useful steps– Detect and discover steps provided by contributed modules

and themes

Tools: Behat how does it work?

Feature: Login

In order to see new personal content

As a visitor

I need to log in

Scenario: Successfully log in

Given I am on the homepage

And I fill in "user" with "hans@ilikedrupal.cafe"

And I fill in "password" with "h3l3m@@lpr1m@"

And I press "sign in"

Then I should see "Success!"

Tools: Mink how does it work?

$session->visit('http://drupalcafe.demo/some_page.php');

echo $session->getCurrentUrl();

echo $session->getStatusCode();

echo $session->getPage()->getContent();

$session->visit('http://my_project.dev/second_page.php');

$session->reload();

$session->back();

$session->forward();

echo $session->evaluateScript(

"return 'something from browser';"

);

$session->wait(

5000,

"$('.suggestions-results').children().length > 0"

);

Tools: Wraith

● Screenshot, or visual diff testing between 2 domains (production and development)

● Regression testing of a design/theme● Uses Imagemagick and PhantomJS for

rendering and comparison● A bit like the distribution testing tool OpenQA.

Tools: Wraith how does it work

Tools: CasperJS

● Scripted testing, a bit like Selenium● Uses PhantomJS (webkit) or SlimerJS (gecko)

– Test responsive layouts with breakpoints on different resolutions– Check placement of DOM elements on a page/response– Test user interactions with the site (authoring, etc) with keyboard

input, mouse clicking, pressing keys (ctrl), etc.

● Makes it possible to functionally compare separate branches with master before merging.

Tools: CasperJS how does it work?

var casper = require('casper').create();

casper.start('http://casperjs.org/', function() {

this.echo(this.getTitle());

});

casper.thenOpen('http://phantomjs.org', function() {

this.echo(this.getTitle());

});

casper.run();

Tools: CasperJS How does it work?

$ casperjs sample.js

CasperJS, a navigation scripting and testing utility for PhantomJS

PhantomJS: Headless WebKit with JavaScript API

Migration, what?

● This doesn't look like a migration● It looks like a lot of shiny new tools!● True, how cool is that? :-)● But you can prepare for these new tools and

also for the migration of your modules (and tests) from Drupal 7 to 8.

CI integration (mostly Jenkins)

● Already possible with D7, but the 3rd party libs in D8 (Symfony) already have a lot of tools/scripts for this.

● Phing Drush Task● Composer● Dependency Injection (see Symfony docs for

the concept) for better staging management.

Configuration Management Initiative

● How does that relate to testing?– Easier staging (OTAP)

● A LOT easier....

– Thanks to separation of configuration and state.

● Risks?– Yes, separate configs could mean different test

circumstances.

Content Staging Initiative

● There are multiple categories of content in any website:– Highly dynamic (news articles, blogs, products)

– Medium dynamic (faqs)

– Semi static (mission statements, organization info, etc)

● Semi static content is often an integral part of the site design and should be tested.

● In D7 this meant database migrations (painful since both configuration, state and content are in the same db).

● Content Staging Initiative strives to enable some forms of content to be stages (OTAP like).– Also see Site Preview System module, preparing revisions as staging.

Features module in D8

● Problem for Features in D7 is that application state, config and content are all in DB.– Not all of them are easily and reliably exportable for Features.

● In D8 config is in code (although cached in db).● State (ie cron, or import status) is no longer a variable.● Content is still entities (DB).● Features will be reduced to only export bundled functionality

(advanced “meta” modules).● This reduction of responsibility also makes it easier to test

Write modular (object oriented) code

● Drupals classical approach is funcitonal– Hooks

● Hooks are tightly coupled into the forms and menu system.– Not necessarily the best abstraction for functional units.

● Actually, implementing business logic into hooks is a recipe for spaghetti code.

● Start writing your logic into objects!

X Autoload

● Add PSR-0 (and 4) style autoloading to Drupal 7!– This will be the standard in D8

● Put your logic into separate classes– This results in cleaner implementation code

– And easier migration to D8, ideally only the hook invoking needs to be reimplemented (events).

– And because of this separation also easier testing!

● Side effect: 3rd party PHP libraries are much easier to use with this.

Composer

● Management system for 3rd party libs● Composer Manager (ie Update manager for

Composer packages)● Composer Autoload (alternative for X Autoload for

Composer packages)● Git wrapper, develop through git, package your

module as composer package, deploy with composer.

● This is THE way to get off the island!

Additional testing

● Integrate security testing into the acceptance testing with OWASP tools like:– ZAP Proxy

● Finding possible vulnerabilities in user interaction data streams.

● Automated request generation.

– Sprajax● AJAX enabled apps testing.

Any questions?

¿