automated white box testing with genrocket


• End to end user experience testing

• Can be manual or automated

Black Box Testing

Functional Testing

Software Testing PyramidCovers

Let’s take a look at the software testing pyramid. Let’s start at the top with functional testing (AKA Black box testing).




Black Box Testing

Functional Testing

Software Testing PyramidCovers

Black box testing covers the end to end user experience. This can be done manually or automated.


Black Box Testing

Functional Testing

Software Testing Pyramid



Covers

Black box testing is important but it only tests 10% of the code base. How do you test the other 90%?

10% code coverage


Black Box Testing

White Box Testing

Unit Testing

Integration Testing

White Box Testing

Functional Testing


90%



Covers

The only way to test the other 90% is with integration and unit tests. These are both types of white box tests.

10% code coverage


White Box Testing

• Business Logic• Service Tier• Micro services• Automated

Unit Testing

Integration Testing

• Units of code• Automated

White Box Testing

Functional Testing

Without white box tests, you can’t test business logic, service tier methods, micro services, or units of code.

The hard thing is that white box testing is complex and time consuming.

Black Box Testing

10% code coverage



CoversSoftware Testing Pyramid

Covers

Covers



• Manual and Automated

Black Box Testing

White Box Testing

10%• Business Logic• Service Tier• Micro services• Automated

Unit Testing

Integration Testing

• Units of code• Automated

White Box Testing

Functional Testing


At GenRocket we have found a way to automate white box test creation. As selenium is to black box testing, GenRocket is to white box testing. 90%

Covers

Covers


We took one test engineer and had him focus on creating white box tests for two clients with GenRocket.

• 1 month of work (20 days)• 40 hours a week• Trained on GenRocket • One application similar to Shopify and

the other is similar to HubSpot

Besides being a software company we also do some testing projects for our clients. We had one our testers create all of the white box tests for two clients. Both of these applications have more than 100 data base tables.


Here’s what he did:

• Created a total of 770 integration tests

• Averaging ~38 Tests a day9/2

1/16

9/22/1

6

9/23/1

6

9/26/1

6

9/27/1

6

9/28/1

6

9/29/1

6

9/30/1

6

10/3/

16

10/4/

16

10/5/

16

10/6/

16

10/7/

16

10/10

/16

10/11

/16

10/12

/16

10/13

/16

10/17

/16

10/18

/16

10/20

/16

24

57

39

62

3430

40

23

37

57

23

51

6973

57

20

4

13

24

33

Tests Created / Day

Here’s what he was able to do…

On his best day he created 73 integration tests!


Here’s what he did:

• Created a total of 770 integration tests

• Averaging ~38 Tests a day

• Beating the average of 2 integration tests a day by ~20x

9/21/1

6

9/22/1

6

9/23/1

6

9/26/1

6

9/27/1

6

9/28/1

6

9/29/1

6

9/30/1

6

10/3/

16

10/4/

16

10/5/

16

10/6/

16

10/7/

16

10/10

/16

10/11

/16

10/12

/16

10/13

/16

10/17

/16

10/18

/16

10/20

/16

24

57

39

62

3430

40

23

37

57

23

51

6973

57

20

4

13

24

33

2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2

Tests Created / Day

Even on his worst day he was 100% faster than the average!

Most test engineers are able to write complex integration tests but they can only write about 2 per day.


A New Type of Tester

A White Box Tester Knows:

So you can’t just take any person and have them generate tests at that speed. They need to be able to know these 5 things.




1. Programming Language

They need to know the programming language of the application. They don’t have to be masters but they have to know the basics!




1. Programming Language2. Application Framework

They need to understand the application framework. Such as, Ruby on Rails, Grails, Node, or Django.




1. Programming Language2. Application Framework3. Database Normalization

Database normalization. The process of organizing the columns and tables of a relational database (Wiki).

https://en.wikipedia.org/wiki/Database_normalization




1. Programming Language2. Application Framework3. Database Normalization 4. GenRocket Test Data Design Patterns5. The GenRocket Platform

These last two are specific to the GenRocket application and to design patterns. We provide training on both of these topics.


What this means for you…

Testers needed per project…

* Capers Jones – The Economics of Software Quality

Let’s take a look at what using GenRocket means for your team. If you have a team of testers…





decreases by 10x.

You can reduce the number of testers on each project because each GenRocket tester is 10X faster.




The application quality…

decreases by 10x.

Since the quality of testing has increased dramatically the application quality…





decreases by 10x.

increases.

Increases! Meaning happier customers and better applications.





decreases by 10x.

increases.

The total cost of ownership…

And since the code quality has increased that means the total cost of ownership…





decreases by 10x.

increases.

The total cost of ownership…decreases 50%*.


Decreases by 50%! This is based off data from Capers Jones. He studied 600 companies and 13,000 development projects over a 5 year cycle and established high quality software decreases total cost of ownership by 50%.





decreases by 10x.

increases.

The total cost of ownership…decreases 50%*.


Decrease in costs, higher quality testing, higher quality applications, and lower costs of ownership…now you’re probably wondering what you need to do to use GenRocket.


Implementation

1. Use an application framework and modularize code

In order to use GenRocket you need to use an application framework. Also, modularize your code so it is testable -- try to keep each function or method between 20-30 lines of code.


Implementation


2. Import and setup your data model

You will then import your data model into the GenRocket Web Application with a DDL file. In GenRocket you will establish the parent child relationships of your model.


Implementation



3. Create and run Scenarios for each Domain

You will then create a scenario for each Domain. A Domain is the equivalent of a table in your database. You will download these Scenarios and run them on your machine to generate white box tests.


Implementation




4. Write custom white box tests

You will then write custom white box tests.

Now you’re probably going to say “Wait a second! Didn’t you say you automate this?”

We do – we automate the hard parts. These hard parts prevent engineers from writing these tests in the first place. The test code we generate helped our tester create white box tests 20x faster.


Implementation 1. Use an application framework and

modularize code



4. Write custom white box tests

5. Run tests and identify bugs

And last, automatically run all the white box tests using the application framework’s built-in test service.

It is also easy to update your tests in GenRocket as your data model changes over time.


Visit GenRocket.com to learn more.

http://www.genrocket.com/

automated white box testing with genrocket

Technology