using fitnesse to test the “rest” of the · pdf filewe are using fitnesse and the...
TRANSCRIPT
![Page 1: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/1.jpg)
Erik Stensland
![Page 2: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/2.jpg)
Abstract
A challenge for QA historically has been to move our testing efforts as deep into the development cycle as possible. Some of our fellow QA‟ers have been fortunate enough to be able to work with development to write Unit tests. However most times, QA is left to test at the UI level with not much time left for items such as Application Security, Unicode and Stability testing. Wouldn‟t it be nice to have a happy medium where most QA organizations with minimal investment could begin testing at a level before the UI but after Unit testing for these areas specifically. Enter FitNesse, FitNesse has been around since 2005 but more recently with the increase in RESTful type services, at least at Pearson eCollege, it is making a big comeback. I will be discussing our approach on how we are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use FitNesse to bring Application Security, Unicode and Stability testing closer to our development teams.
![Page 3: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/3.jpg)
Topics
Development Cycle
Pearson eCollege Benefits
FitNesse
REST
Types of REST tests
Functional
Integration
App Security
Unicode
Stability
![Page 4: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/4.jpg)
Traditional
Development Cycle
• Service Development
Unit Testing
• UI Development
Functional Testing
• Integration Development
Integration Testing
• Tuning Development
AppSec/Unicode/Stability
![Page 5: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/5.jpg)
Non Traditional Dev
Cycle
Stability Testing
Functional Testing Integration Testing
Application Security
Testing Unicode Testing
Service Development
![Page 6: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/6.jpg)
Pearson eCollege Benefits
Faster development and test time – Increased releases to production. Set an eCollege record in July for number of releases.
30,000+ validations for a brand new product with first line of code developed in April and released to production in mid-July.
Development and SQE both take ownership in the tests.
Development can run the tests while SQE focuses on new tests using Jenkins.
SQE now provides a huge amount of information to the development process quicker then ever before.
Test cases are now self documenting.
Extremely easy barrier to entry. In less then 12 months we have increased from 1 team utilizing FitNesse to ~10 teams generating 10s of thousands of validations.
![Page 7: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/7.jpg)
A quote from our CTO!
Recently our CTO blogged the following:
“Looking at just Web Services on Demand and some new admin services we have over 21,000 automated validations, testing all the capabilities of several hundred web services in all their permutations. This means that we can test every single service we have built to date on these projects with the click of a button and a few minutes of time. Power. Real Power. Further, we are using open source tools to do the testing. Free. Even further, our team is on the cutting edge. We are extending the open source frameworks and sending the code back to the projects. Innovative. Almost no one has done what we are doing before, and those that have are the likes of Google, Amazon & Twitter.”
![Page 8: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/8.jpg)
FitNesse ???
Fully integrated standalone wiki, and acceptance testing
framework.
Tool for enhancing collaboration in software
development.
Compares customer expectations to actual
results.
Invaluable way to have development and
SQE collaborate on complicated
requirements. (i.e. Test Driven Development)
![Page 9: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/9.jpg)
Architecture
SUT
Test Cases
FitNesse
REST Fixture
REST Services
![Page 10: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/10.jpg)
Easy Barrier to Entry
FREE
Easy to setup
No special hardware.
Wiki web server
Simply use your browser to create, edit and execute tests.
Quick Learning curve.
![Page 11: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/11.jpg)
Nuts and Bolts
Global Variables
!define var1{Leonardo Da Vinci}
Local Variables
|let|$var2|body|/root/data/name/text()| |
Test Tables
Input and Expected output.
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": "-!${var1}!-"}-!|
|POST|/create|200|||
|let|$var2|bodyregex|!-Object (.*) was-!| |
![Page 12: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/12.jpg)
“REST” web services
Three defined aspects
Base URI: http://localhost:8082/
Internet media type: JSON, XML
Set of Operations: GET, POST,
PUT, DELETE
Representational State Transfer
Style of software architecture for distributed hypermedia.
Scalability, Generality of interfaces, independent
deployment of components.
![Page 13: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/13.jpg)
RestFixture
Custom Fixture that allows developers, quality and/or
product owners to write test tables for REST services
with simplicity in mind. The idea is to write tests that are
self documenting and easy to write and read, without the
need to write Java code.
Open Source- https://github.com/smartrics/RestFixture
Pearson eCollege - https://github.com/tfredrich/rest-
fixture
![Page 14: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/14.jpg)
Types of “REST” tests.
Functional
JSON, XML, Special Characters, Tunneling, Empty fields, Null fields, Authorization, Invalid
Integrated
User Scenarios / Work Flows
App Sec
Cross-site scripting / SQL injection
Unicode
Different languages
Stability
Timing
![Page 15: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/15.jpg)
What is Functional ?
Functional Service testing is the verification that the SUT
works according to specifications and that it handles
erroneous actions and data correctly
Positive
Negative
JSON and XML
Invalid Fields
Tunneling (?_method=PUT)
Special Characters (@#$%^&*)
![Page 16: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/16.jpg)
Functional Example
STANDARD POST
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": "erik"}-!|
|POST|/create|200||//status/text()='success'|
STANDARD READ
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|GET|/read|200|Content-Type : application/json |!-
//status/text()='success'
//code/text()='200'
//message/text()=‟Successful Read‟-!|
![Page 17: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/17.jpg)
What is Integrated ?
Integration Testing is when individual modules are combined
and tested as a group.
User Scenarios
Work Flows
![Page 18: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/18.jpg)
Integrated Example
STANDARD POST
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": ”Erik"}-!|
|POST|/create|200||//status/text()='success'|
|let|$var2|bodyregex|!-Object (.*) was-!| |
URLENCODE
|!-org.eclg.fitnesse.tools.StringFixture-!|
|value|=urlencode()|
|%var2%|var2|
READ USING VARIABLE FROM POST
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|GET|/correct/%var2%|200|Content-Type : application/json |!-
//status/text()='success'
//code/text()='200'
//message/text()='YOU ARE SUCCESSFUL'
-!|
![Page 19: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/19.jpg)
What is AppSec ?
Malicious Code Injections
Penetration Testing
Input Validation
Variable Manipulation
AppSec evaluates the security posture of an application across
the development life cycle, enabling you to identify, eliminate,
and prevent security risks in the applications that drive your
business.
![Page 20: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/20.jpg)
AppSec Example
APP SECURITY
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": "<script>alert(document.cookie);</script>"}-!|
|PUT|/appsec|200||//name/text()='alert(document.cookie);'|
![Page 21: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/21.jpg)
What is Unicode ?
Spanish
Russian
Chinese
Unicode testing is testing character sets that define every
character in most of the speaking languages in the world.
![Page 22: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/22.jpg)
Unicode Example SPANISH
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": -!"Puedo comer vidrio, no me hace dao"!-}-!|
|PUT|/unicode|200||//name/text()=„Puedo comer vidrio, no me hace
daño‟|
RUSSIAN
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": -!"Съешь же ещѐ этих мягких французских булок да
выпей чаю"!-}-!|
|PUT|/unicode|200||//name/text()=„Съешь же ещѐ этих мягких
французских булок да выпей чаю‟|
CHINESE
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": -!"我能吞下玻璃而不伤身体"!-}-!|
|PUT|/unicode|200||//name/text()=„我能吞下玻璃而不伤身体‟|
![Page 23: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/23.jpg)
What is Stability ?
Stability testing checks to see if the service can continuously
perform with in an expected time range.
Timing
![Page 24: Using Fitnesse to test the “REST” of the · PDF filewe are using FitNesse and the REST fixture to test standard CRUD functionality but more importantly how we are starting to use](https://reader033.vdocument.in/reader033/viewer/2022051305/5a89806c7f8b9a78648b496d/html5/thumbnails/24.jpg)
Stability Example
STABILITY
|!-smartrics.rest.fitnesse.fixture.RestFixture-!|${trainingHost}|
|setBody|!-{"name": "erik"}-!|
|startTimer|
|PUT|/test|200||//name/text()=„erik‟|
|checkTimer|20|50||