behavior driven testing of web services
TRANSCRIPT
+
Sreedevi VedulaShraddha Suman
Five Problems
Behavior Driven Testing
For Web Services
Five Patterns`
+The five problems
Can’t name my data? The objects are just exploding! Is there a way I can
reuse them? How deep can this get? Unhappy path testing is truly a unhappy one! The cleanup is a problem as always!
+An Example
Inventory Management Application Create categories Create brands Add products to the inventory Search for products View inventory summary
Disclaimer: All applications used in this presentation are fictitious and
have no code or existence outside of this presentation. Any resemblance to client project or internal project is only a co-incidence.
+1. Can’t name my data?The Problem
Given I create the below categories in inventory | category_id | category_name | | 18765 | Music Players | | 18783 | TV |And I create the below brands in inventory | name | brand_id | | Samsung | 195256 | | Philips | 197654 | And I create the below products in Inventory | brand_id | product_id | category_id | model_id | | 197654 | 197654_10987 | 18765 | E4567 | | 195256 | 197654_11234 | 18765 | M7854 | | 197654 | 195256_11123 | 18783 | R4398 | | 195256 | 195256_10621 | 18783 | V4563 |When I filter the products in the inventory | query_param | value | | brand_id | 195256| | category_id | 18783 |Then the below products should be returned | brand_id | product_id | category_id | model_id | | 195256 | 195256_10621 | 18783 | V4563 |
+Difficulties
Uniqueness Violation if the tests re-run Readability Suffers, difficult to review Hard-coded values obscure cause-effect relationships Numbers might collide with previously generated data
+Use Data TemplatesThe Pattern Applied
Given I create the below categories in inventory | category_id | category_name | | <category_id_1>| Music Players | | <category_id_2>| TV |And I create the below brands in inventory | name | brand_id | | Samsung | <brand_id_1>| | Philips | <brand_id_2>| And I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id_2> | <product_id_1> | <category_id_1> | <valid-model-id> | | <brand_id_1> | <product_id_2> | <category_id_1> | <valid-model-id> | | <brand_id_2> | <product_id_3> | <category_id_2> | <valid-model-id> | | <brand_id_1> | <product_id_4> | <category_id_2> | <valid-model-id> |When I filter the products in the inventory | query_param | value | | brand_id | <brand_id_1> | | category_id | <category_id_2> |Then the below products should be returned | brand_id | product_id | category_id | model_id | | <brand_id_1> | <product_id_4> | <category_id_2> | <valid-model-id> ?!? |
+Plumbing
Create data templates in code and generate relevant data in code for each template encountered.
Maintain a global test data map and store the test data values against the templates. A subsequent request for the same template should
retrieve this value from the map instead of creating new data.
+2. Object Explosion The Problem
Given I create the below categories in inventory | category_id | category_name | | <category_id_1>| Music Players | | <category_id_2>| TV |And I create the below brands in inventory | name | brand_id | | Samsung | <brand_id_1>| | Philips | <brand_id_2>| And I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id_2> | <product_id_1> | <category_id_1> | <valid-model-id> | | <brand_id_1> | <product_id_2> | <category_id_1> | <valid-model-id> | | <brand_id_2> | <product_id_3> | <category_id_2> | <valid-model-id> | | <brand_id_1> | <product_id_4> | <category_id_2> | <valid-model-id> |When I filter the products in the inventory | query_param | value | | brand_id | <brand_id_1> | | category_id | <category_id_2> |Then the below products should be returned | brand_id | product_id | category_id | model_id | | <brand_id_1> | <product_id_4> | <category_id_2> | <valid-model-id> ?!? |
+Difficulties
No way to reuse objects No way to access some properties of the objects like
object[property]
+Name Your ObjectsThe ref_name case - The Pattern Applied
Given I create the below categories in inventory | category_id | category_name | | <category_id_1>| Music Players | | <category_id_2>| TV |And I create the below brands in inventory | name | brand_id | | Samsung | <brand_id_1>| | Philips | <brand_id_2>| And I create the below products in Inventory | ref_name | brand_id | product_id | category_id | model_id | | <product_1> | <brand_id_2> | <product_id_1> | <category_id_1> | <valid-model-id> | | <product_2> | <brand_id_1> | <product_id_2> | <category_id_1> | <valid-model-id> | | <product_3> | <brand_id_2> | <product_id_3> | <category_id_2> | <valid-model-id> | | <product_4> | <brand_id_1> | <product_id_4> | <category_id_2> | <valid-model-id> |When I filter the products in the inventory | query_param | value | | brand_id | <brand_id_1> | | category_id | <category_id_2> |Then the below products should be returned | ref_name | | product_4 |
+Name Your ObjectsThe Of Case - The Pattern Applied
Given I create the below categories in inventory | ref_name | category_id | category_name | | category_1 | <unique-category-id>| Music Players | | category_2 | <unique-category-id>| TV |And I create the below brands in inventory | name | brand_id | | Samsung | <brand_id_1>| | Philips | <brand_id_2>| And I create the below products in Inventory | ref_name | brand_id | product_id | category_id | model_id | | <product_1> | <brand_id_2> | <product_id_1> | <category_id of category_1> | <valid-model-id> | | <product_2> | <brand_id_1> | <product_id_2> | <category_id of category_1> | <valid-model-id> | | <product_3> | <brand_id_2> | <product_id_3> | <category_id of category_2> | <valid-model-id> | | <product_4> | <brand_id_1> | <product_id_4> | <category_id of category_2> | <valid-model-id> |When I filter the products in the inventory | query_param | value | | brand_id | <brand_id_1> | | category_id | <category_id of category_2> |Then the below products should be returned | ref_name | | product_4 |
+Plumbing
Store the objects in the global test data map with ref_name as the key and object properties as key-value pairs.
Build a separate data template which matches “of” templates, make the template look up the global test data map and lookup the properties.
+3. How deep can this get?The Problem
{ "inventory":
{ "brands": [
{ "brand_id": "18456",
"products": [
{ "model_id": "E1234", "quantity": 5 }, { "model_id": "E7654", "quantity":
10 } ] }
{ "brand_id": "17654",
"products": [
{ "model_id": "E2345", "quantity": 7 }, { "model_id": "E8765", "quantity":
17 } ] }
]
}
}
+Difficulties
Nested Payloads cannot be handled in one step as Gherkin only allows two-dimensional tables.
+Break Down Your PayloadsThe Pattern Applied When I view the inventory summary
Then I should see the below brands listed in the summary| brand_id || <brand_id of brand_1> || <brand_id of brand_2> |
And I should see the below products for brand <brand_1>| model_id | quantity || <model_id_1> | 5 || <model_id_2> | 10 |
And I should see the below products for brand <brand_2>| model_id | quantity || <model_id_3> | 7 || <model_id_4> | 17 |
+Plumbing
Create steps for each nesting level and make assertions at each level in the payload.
+4. Unhappy path testing The Problem
When I create the below products in Inventory | brand_id | product_id | category_id | model_id | | language specific null value| 12345_10987 | 1052 | E4567 |Then I should see the below error message “brand_id cannot be null”
When I create the below products in Inventory | brand_id | product_id | category_id | model_id | | 12345 | An Array | 1052 | E4567 |Then I should see the below error message ”product_id should be an integer”
When I create the below products in Inventory | brand_id | product_id | model_id | | 12345 | 12345_10987 | E4567 |Then I should see the below error message “category_id is a mandatory parameter”
And then repeat the above scenarios for every property
+Difficulties
Cannot specify language specific invalid values like null, None
Challenging to specify empty string in Gherkin Cannot distinguish between different data types Missing properties would require different scenarios for
each property
+Use Scenario Outlines & Negative TemplatesInvalid Values Case - The Pattern Applied
Scenario Outline: Cannot create products with missing propertiesWhen I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id> | <product_id> | <category_id> | <model_id> |Then I should see the below error message “<error-message>”
Examples: | brand_id | product_id | category_id | model_id | error-message || <null-value> | <product_id_1> | <category_id_1> | <model_id_1>| brand_id cannot be null. || <empty-string> |<product_id_2> | <category_id_1> | <model_id_2>| brand_id cannot be empty. || <brand_id_1> | <negative-int> | <category_id_1> | <model_id_1>| product_id can’t be negative.|
+Use Scenario Outlines & Negative TemplatesInvalid Data Types Case - The Pattern Applied
Scenario Outline: Cannot create products with missing propertiesWhen I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id> | <product_id> | <category_id> | <model_id> |Then I should see the below error message “<error-message>”
Examples: | brand_id | product_id | category_id | model_id | error-message || <random-array>| <product_id_1> | <category_id_1> | <model_id_1> | brand_id must be integer. || <random-map> |<product_id_2> | <category_id_1> | <model_id_2> | brand_id must be integer. || <brand_id_1> | <random-string>| <category_id_1> | <model_id_1>| product_id must be integer. |
+Use Scenario Outlines & Negative TemplatesMissing Properties Case - The Pattern Applied
Scenario Outline: Cannot create products with missing propertiesWhen I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id> | <product_id> | <category_id> | <model_id> |Then I should see the below error message “<missing-property> is mandatory.”
Examples: | brand_id | product_id | category_id | model_id | missing-property || <missing> | <product_id_1> | <category_id_1> | <model_id_1> | brand_id || <brand_id_1> | <missing> | <category_id_1> | <model_id_2> | product_id || <brand_id_1> | <product_id_1> | <category_id_1> | <missing> | model_id |
+Plumbing
Define data templates such as <random-array>, <negative-int> etc., to provide sample values of the data type in code.
Remove properties for the <missing> templates in code and submit the payload.
+5. The cleanup problem!The Problem
Background:
When I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id> | <product_id> | <category_id> | <model_id> |
Scenario:
When I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id> | <product_id> | <category_id> | <model_id> |
+Difficulties
Background is usually used (tweaked) to create objects once for the entire feature. But, we cannot really know when the object is created. Can we?
+Track Objects and their LifetimeThe Pattern Applied
Background:
When I create the below products in Inventory once for the entire feature | brand_id | product_id | category_id | model_id | | <brand_id> | <product_id> | <category_id> | <model_id> |
Scenario:
When I create the below products in Inventory | brand_id | product_id | category_id | model_id | | <brand_id> | <product_id> | <category_id> | <model_id> |
+Plumbing Build explicit steps for feature level setups and track Object Lifetime
in the steps
@step('I create the below products in inventory once for the entire feature’)def add_products_for_feature(context): if not DataManager.products_created_for_feature: DataManager.products_created_for_feature = True add_products(context, "feature”)
@step('I create the below products in inventory’)def add_products (context, scope=“scenario”): // Create Products
DataManager.track_objects_for_teardown(products, scope)
Delete objects at “feature” scope after the feature execution and delete objects at “scenario” scope after the scenario execution.
+
That is all! Hope you liked it!
+
Thank You!