Download - Automated Scenario Test
-
8/14/2019 Automated Scenario Test
1/27
Purpose
This document provides guidance and information that is necessary to be able to effectively
implementing the automations of the test-cases.
Audience
The audience of this documentation is the software SDETs1 responsible for the implementation of the
automations of the test-cases.
Implementing the automations involves translating the test-cases into XML documents where the steps
in the test case are spelled out, and implementing custom actions (in the java programming language)
that actually carry out the tasks to in each step. Therefore, knowledge of XML and programming injava are required on the side of the SDET.
Scope
Each XML mentioned above captures a test-case (the intention and the steps) in a program-readableformat. Throughout this documentation we will refer to that XML as scenario xml. This documentation
explains all the elements and attributes expected / allowed in the scenario XML.
1 Software Development Engineer in Test
-
8/14/2019 Automated Scenario Test
2/27
Scenario XML
The scenario XML will be explained using several examples, starting with the one below.
eq(mediator("isStarted()"), true)
"lee"
step 1 desc"fu"
action 1.1 desclist("a","b")range(7,8,1)9
action 1.2 desc
{rand = new Random(action.getArgument("randSeed"));}Integer theNextInt = new Integer(rand.nextInt());
action.getStep().setValue("theNextInt", theNextInt);System.out.println(Thread.currentThread() + ": Done! : " + theNextInt);return null;]]>list(10000, null)
2action 2.1 desc
list(3, true, "test", null)switch(ref(x), {true:concat("It's true! Look:", ref(x)),
"zeppelin":"Wow that's a monster.", any:"Well..."})
Insert 1. Scenario XML #1
-
8/14/2019 Automated Scenario Test
3/27
First pass through the scenario XML
Thenode is the root of a scenario XML. It has an optional attribute repetition, whose
value can be any integer equals to or greater than 1 (by default it is set to 1). It is used to specify how
many times the scenario will be ran, sequentially. If we set it to 0, the scenario will be repeated overand over, until we kill the process.
Another optional attribute ofismax-thread, that specifies the maximum number of threads
the scenario runner can use to run the actions. The number specified here is not a hard limit; there can
be a little more threads than the number specified. As a general guideline, in order to avoid resorting
into sequential execution of actions, pick the highest multiplicity in the scenario, add a constant to it,and assign the result to themax-threadattribute. In the above example it's 4 (the multiplicity ofstep 2),
and the constant used is 1 (becausestep 2 is one level below the scenario). Therefore the chosen value
formax-threadis 4 + 1 = 5.
Inside thenode we can put anode, in which we write the (high-level) description
of the scenario. We must enclose the description in a CDATA section if the description contains special
characters which otherwise would render the scenario XML unprocessable.
A scenario, as well as step and action, is a configurable node; it can take arguments. Those arguments
are contained in thenode . Each of them is represented by annode , whose value can
be a string literal, integer literal, bigdecimal literal (e.g.: 1.234), boolean literal (true / false), null, XMLrepresentation of a java object, XML tree, or function.
The following table contains shows some examples of how to define an argument.
-
8/14/2019 Automated Scenario Test
4/27
Type Example Value received in the java program(action class)
String literal foobar java.lang.String foobar
Int literal 78 java.lang.Integer 78
Boolean literal false java.lang.Boolean false
BigDecimal literal 7.5 java.math.BigDecimal 7.5
Null2 null An instance of com.baktun.bstrd.director.Null
Any any An instance of com.baktun.bstrd.director.Any
XML tree
Some beans
An instance of org.w3c.dom.Node
XML representation ofJava object
Some beans
An instance ofcom.eshop.PurchaseOrder3
Function4 mult(3,2) An instance of BigDecimal 6
Table 1. Examples of argument
The arguments defined in a configurable node are visible to the child configurable-nodes. The usual
scoping rules apply (i.e.: an argument defined in a configurable will override the argument with thesame key defined in the parent node(s)).
2 Assigning nullto an argument is not the same as not defining the argument.
3 The class must contain special (JAXB) annotations that would allow automatic mapping to XML .
4 All the available functions will be explained in separate section.
-
8/14/2019 Automated Scenario Test
5/27
A scenario is composed of one or more steps, which will be executed one after another. Those steps
must be contained within thenode . Each of them is represented by anode that
has the following attributes:
name (mandatory): the name of the step. Each step within the same scope5 must have a unique
name.
offset (mandatory): the duration, in seconds, for the execution of the step to be delayed. The
value must be an integer equal to or greater than 1. mult (mandatory): the number of instances of the step to be created and executed. The value
must be an integer equal to or greater than 1.
concurrent (optional): specifies whether the instances of the step will be executed sequentially
or concurrently. The valid value is true (for concurrent) orfalse (for sequential). The defaultvalue isfalse.
Inside the a step we define the actions that belong to the step. Those actions must be contained within
thenode. Each of them is represented by annode that has the following
attributes:
name (mandatory): the name that uniquely identifies the action within the step.
class (optional): the name of the java class that actually performs the task intended by theaction. If we don't specify this, we must write the code directly inside the script node , as
exemplified by action 1.2.
asynch (optional): specifies whether the action is to be executed in the same thread as the
containing step or in separate thread.
Running a scenario
To see a scenario XML in action, a separate program has to be written. This program, which we will
refer to as directorthroughout this documentation, in general performs the following tasks:
Load the scenario XML (into a java object of type ScenarioDefinition).
Create an instance ofmediator.
Create an instance ofScenarioRunner.
Start the scenario runner by calling its run(...) method6, feeding in the scenario definition
created in step #1.
Start the mediator.
The mediator is a generic java object; it can be anything. It is the entity we want to control, manipulate,
or test throughout the execution of scenario, by means of actions. The following diagram depicts the
relationship between scenario, mediator, and action.
5 The meaning of it will be clarified in the section where action-nesting is explained.
6 It has to run in a thread different from the one that runs the mediator.
-
8/14/2019 Automated Scenario Test
6/27
Figure 1. The relationship between scenario, mediator, and actions.
The following insert contains an example of such director. In this example, the mediator is an instanceofcom.baktun.bstrd.unittest.Frijol7.
We have something quite similar for the spectel bridge simulator; the director is a java applicationnamed com.baktun.bridgesimulator.BridgeDirector, and the mediator is an instance of
com.baktun.bridgesimulator.Bridge, wrapped inside an instance of
com.baktun.bridgesimulator.BridgeServer.
package com.baktun.bstrd.unittest;
import java.io.BufferedReader;import java.io.FileReader;import org.apache.log4j.Logger;import com.baktun.bstrd.director.ScenarioDefinition;import com.baktun.bstrd.director.ScenarioRunner;
publicclass FrijolDirector {privatestaticfinal Logger logger = Logger.getLogger(FrijolDirector.class);
private FrijolDirector() {}
publicstaticvoidmain(String[] args) throws Exception { if (args == null || !(args.length == 2) ) {
System.out.println("Usage: java FrijolDirector [keep-frijol-up]");System.exit(-1);
}
boolean keepFrijolUp = true; if (args.length == 2) {
keepFrijolUp = Boolean.parseBoolean(args[1]);}
String scenarioDefXml = null;{BufferedReader br = new BufferedReader(new FileReader(args[0]));StringBuffer sb = new StringBuffer();
try {String line = br.readLine();
while (line != null) {sb.append(line).append("\n");line = br.readLine();
7 Spanish word for bean.
-
8/14/2019 Automated Scenario Test
7/27
}} finally {
if (br != null) {br.close();
}}
scenarioDefXml = sb.toString();}
final ScenarioDefinition scenarioDef = ScenarioRunner
.loadScenarioDef(scenarioDefXml);
final Frijol frijol = new Frijol();
final ScenarioRunner scenarioRunner = new ScenarioRunner();
Thread scenarioRunningThread = new Thread(new Runnable() { publicvoidrun() { try {
scenarioRunner.run(scenarioDef, frijol);} catch (Exception e) {e.printStackTrace();
}}
});
scenarioRunningThread.setPriority(Thread.MIN_PRIORITY);
scenarioRunningThread.start();
frijol.run(); }
}
Insert 2. An example director
The output of the execution of the director above, in the following insert, will help clarifying a coupleof important points about the execution model of the scenario runner.
2009-09-23 15:42:24,564 [INFO] Thread-2 com.baktun.bstrd.unittest.Frijol - Frijol is goingto sleep for 5 seconds....
2009-09-23 15:42:24,583 [INFO] Thread-1 com.baktun.bstrd.director.ScenarioRunner -Running scenario for the 1 times2009-09-23 15:42:30,640 [INFO] Thread-1 com.baktun.bstrd.director.ScenarioRunner -About toexecute step step 1
Thread[pool-1-thread-2 ,5,main]:{jet=lee, kung=fu, a=b , c=9, b=8 }
Thread[pool-1-thread-1 ,5,main]:{jet=lee, kung=fu, a=a , c=9, b=7 }------2009-09-23 15:42:30,661 [INFO] Thread-1 com.baktun.bstrd.director.ScenarioRunner -About toexecute step step 2
Thread[pool-1-thread-3 ,5,main]: Done! : 1913434059
Thread[pool-1-thread-4 ,5,main]: Done! : -498702880
Thread[Thread-1 ,1,main]:{jet=lee, y=Well..., x=3}---Thread[Thread-1,1,main]:{jet=lee, y=It's true! Look:true, x=true}---Thread[Thread-1,1,main]:{jet=lee, y=Well..., x=test}---Thread[Thread-1,1,main]:{jet=lee, y=Well..., x=null}---
Insert 3. Output of the execution of example director
-
8/14/2019 Automated Scenario Test
8/27
Threading in the scenario
Sincestep 1 is declared as concurrent, there mightbe new thread(s) allocated for the execution of the
instance(s) ofstep 1. In the above case, we also declaredstep 1 to have multiplicity 2, which might
cause two instances ofstep 1 to be created, each of them will be running in separate threads8.
In contrast, when a step is not declared as concurrent, as is the case ofstep 2, it will be executed in thesame thread as the one in which the containing node is running . In the above case, the containingnode is the scenario, which is running in the main thread.
The scenario runner will go through all the steps sequentially, in the document order. That means,step
2 will be executed only after all the instances ofstep 1 have finished running.
Action will be running in the same thread as the one in which the containing step is running , except
when it is declared as asynchronous (the case of action 1.2). Asynchronous action will be running in
yet separate thread 9.
Implementions of action are not expected to either create or manage any threads (although currentlythere's nothing that prevents it from doing so). If an action is intended to run in the background (e.g.: as
a listener), we can simply declare it as asynchronous in the scenario XML.
The declarative style of doing things here is aimed at improving the communication among the team-
members, by making the intention spelled out clearly in the scenario XML. Above all, this test
framework is a communication tool that fills the gap between the test-case document, and the
executable that realizes the test-case.
The test-framework takes common, repeated concerns (such as threading) away from us, while still
making it accessible in another way (declarations in the scenario XML). This allows us to have a lean
and compact implementation of actions, which are expected to consist of only a few lines of code thatbasically take the arguments passed-in to it, process them a little bit, and use them during the
interaction with the mediator.
8 There is no guarantee that instances of a step declared as concurrent will be running concurrently. The scenario-runnermakes its best effort to fulfill it, though, but if the maximum number of threads is already reached, the scenario-runner
will resort to executing those steps in the same thread as the one in which the parent node is executing, resulting in
sequential execution of those instances of step.
9 Care has been taken about creation of new threads. The engine utilizes Java's built-in thread-pooling mechanism. This
has consequences such as: action that uses ThreadLocal as storage will likely break.
-
8/14/2019 Automated Scenario Test
9/27
Argument's-value resolution
Initially there was a simple mechanism: you declare the argument by simply specifying the key and the
value, what you see (in the XML) is what you get (in the action). Things changed when we introduceddynamism into the scenario XML, by means of a domain-specific expression language.
Some of the functions available in the expression language allows the value of an argument to be basedon the value of other argument, for example: concat(ref(segment_id), "-",
ref(participant_id)).
Now we can make the value of an argument to vary according to the index in the iteration:list("white", "red", "black").
We can also make use of a construct similar to the switch-case-default statement in programming
language like java: switch(ref(hobby), {"music":"give him a guitar",
"literature":"buy him a kindle", null:"give him some cash"}).
By combining that dynamic argument-value and multiplicity of steps, a whole set of combinations ofinputs for the test-case can be packed into a single scenario XML.
There are two functions that particularly deserves a couple of paragraphs (of clarification) here:
The list function, with syntax: list(element_1, element_2, ..., element_n)
The range function, with syntax: range(start_val, end_val, step)
Assigning a list to an argument in the scenario XML does not mean that an instance of
java.util.Listwill be passed-in to the action. Instead, the action will only see a single element
bound to that argument, taken from the list. Example:
list("green", "orange")
list(1980, 1990)
Insert 3. An example scenario definition with list(...)
In the example presented above, there will be two threads. The value of the arguments accessible from
the action running in the first thread: {var_a: green, var_b: 1980} . The second thread, on the other
hand, will see the following values: {var_a: orange, var_b: 1990}.
-
8/14/2019 Automated Scenario Test
10/27
The function range behaves in similar fashion. In fact, we can see a range as just anotherlist, only
that we don't have to specify all the elements manually. That is to say:
range(10, 15, 1) is equivalent to list(10, 11, 12, 13, 14, 15)
range(10, 15, 2)10is equivalent to list(10, 12, 14)
The number of elements in the list must be at least equal to multiplicity of the step to which it directly
belongs. In the above example, list var_a and var_b both belong to the stepstep 1 whose multiplicityis set to 2. Therefore the length of those list be at least equal to 2. Extra elements at the end will simply
be ignored / unused.
The following insert contains another example that uses list, this time involves nesting of actions. In
that case, the length of list var_c and var_d must be at least 1, which is the multiplicity ofstep 2 to
which those lists directly belong.
list("green", "orange")
list(1980, 1990)
list(9)
list("bar")
Insert 4. Another example scenario definition with list
10 Probably mod-check is necessary to eliminate misunderstanding / mis-expectations.
-
8/14/2019 Automated Scenario Test
11/27
-
8/14/2019 Automated Scenario Test
12/27
mult(1.6, 2), for example).
switch: takes the formswitch(selector, map_of_cases), returns the value of the key-value pair
whose key matches the value of the selector. Examples:
switch(ref(x), {"boy": "Rama", "girl": "Shinta", any:
ref(generic_name)}) returns:
the string "Rama" ifx resolves to "boy".
the string "Shinta" ifx resolves to "girl". the value the variable generic_name resolves to ifx resolves to any value other than
"boy" or "girl".
neg: accepts either integer / bigdecimal / boolean, and returns the negated value of its input.Examples:
neg(1) returns Integer(-1)
neg(-2.5) returns BigDecimal(2.5)
neg(ref(x)) returns Boolean(true), assuming x resolves to false.
eq: compares its two inputs, returns true if they're equal. Except for some special cases, the
equality is based on the value of the two inputs (i.e.: the result of calling equals method on
one of the input, passing in the other input as the argument). Examples: eq(1, 1) returns Boolean(true).
eq(1, 1) returns Boolean(false).
eq(1, ref(x)) returns Boolean(true), assuming x resolves to Integer(1).
eq(ref(x), any) returns Boolean(true), assuming x resolves to any non-null value.
eq(mediator(getWatcher()), null) returns Boolean(true), assuming the
invocation of method getWatcher() on the mediator returns null.
eq(mediator(getWatcher()), any) returns Boolean(true), assuming the
invocation of method getWatcher() returns any non-null value.
gt: compares its two inputs, returns Boolean(true) if the first input is greater than the second
input. Will return error if any of the input is neither an instance ofInteger norBigDecimal.Examples:
gt(2, 1) returns Boolean(true).
gt(2.1, 1) returns Boolean(true).
gt(1, 2) returns Boolean(false).
gt(ref(x), 2) returns Boolean(true), assuming x resolves to any Integer or
BigDecimal greater than 2.
gt(2, 1) throws an error.
lt: compares its two inputs, returns Boolean(true) if the first input is smaller than the second
input. Will return error if any of the input is neither an instance ofInteger norBigDecimal.
Examples: lt(1, 2) returns Boolean(true).
lt(1, 2.1) returns Boolean(true).
lt(2, 1) returns Boolean(false).
lt(2, ref(x)) returns Boolean(true), assuming x resolves to any Integer or
BigDecimal greater than 2.
gt(1, 2) throws an error.
-
8/14/2019 Automated Scenario Test
13/27
and: performs logical AND operation on all11 of its inputs. All of the inputs must return an
instance ofBoolean. Examples:
and(true, true) returns Boolean(true).
and(true, false) returns Boolean(false).
and(true, true, ref(x)) returns Boolean(false), assuming x resolves toBoolean(false).
and(mediator(isStarted()), mediator(isOkToTest())) returnsBoolean(true) assuming the invocation of both isStarted() and isOkToTest()
method on the mediator returns true.
and(true, mediator(getNumberOfLines())) throws an error, assuming the
invocation ofgetNumberOfLines()method on the mediator returns anything other than
Boolean (object / primitive).
or: similar to the and function, only that the logical operation performed on the inputs is OR.
rand: takes two inputs, both of them must resolve to integers; will generate a random integer
between the range specified by those two inputs. Examples:
rand(3, 7) can return any of the following values: 3, 4, 5, 6, or 7.
rand(2, ref(x)) can return either 2, 3, 4 assuming x resolves to 4. mediator: takes the OGNL12expression that will be applied on the mediator. The simplest way
to see it: it allows us to specify the chain of method invocations (that starts on the mediator).
This function returns the value returned from the invocation(s). Example:
mediator(isStarted()), invokes the isStarted() method on the mediator, returns
the value returned from that invocation.
mediator(getParticipant(#participant_id)), invokes the
getParticipant(...)method on the mediator, passing in the value of the argument
whose key is participant_id. This requires an argument to be defined in the same node
(action / step / scenario) or above as the node that in which the mediator function is used.
Example:
mediator(getParticipant(#participant_id))part_2009_xx_00021
mediator(getParticipant(#participant_id).getName()), invokes the
getName() method on the object returned by the invocation ofgetParticipant(...)
method on the mediator.
mediator(launchRocket()) throws an error if the method launchRocket() is
neither defined nor accessible in the mediator object.
context: takes the form context(path, context_var_key) . Thepath specifies the node in thescenario where the object stored under the key context_var_key13 is to be found. The syntax of
the path is as follows: . (single dot): current node.
... (triple dot): root node (scenario)
.. (double dot): parent
./action_1
11 There is no short-circuiting here; each and every arguments will be evaluated.
12 Object-Graph Navigation Language: http://en.wikipedia.org/wiki/OGNL
13 Storing object in the scenario-execution context will be explained in separate section.
http://en.wikipedia.org/wiki/OGNLhttp://en.wikipedia.org/wiki/OGNL -
8/14/2019 Automated Scenario Test
14/27
../..
../action_1
../../../action_1
error: it is meant to be used only inside thenode, to specify the error that is
expected to occur inside an execution-node. This function has two possible syntaxes:
with one argument: error(exception_class_name), which will return true if any
instance of error of the specified type occured. Examples: error(java.lang.Exception)
error(java.lang.IOException)
with two arguments: error(exception_class_name, ognl_expression), which
goes further than the first version, by doing a chain of method invocation starting at the
instance of exception (using the specified ognl_expression). This functions returns the valuereturned from the method invocation(s). Examples:
error(java.io.FileNotFoundException, getMessage()); returns the
message of the instance of the FileNotFoundException.
This version oferror function is meant to be used together with the eq function. Example:
eq(c:\\theimportantfile.txt can not be found,error(java.io.FileNotFoundException, getMessage()))
stepidx: this function returns the index of the instance of the step. It takes as input the path to
the execution-node where the calculation of the index is based on14. For example, when we have
a step declaring the following, the second instance of the step will see the integer 2 assigned tothe argument the_index.
...step 1 descstepidx(".")
...
...
Caution about the context_path that is used as input for this function: it must resolve to a
node that is a step. Otherwise an error will be thrown. Example:
...step 1 desc
stepidx(".")...
...
...
...
14 See the explanation of function context for some examples of path.
-
8/14/2019 Automated Scenario Test
15/27
The following definition of argument the_index, however, will not throw an error. Instead,
the action that is part of the second instance of the step will see the integer 2 assigned to the
argument the_index.
...step 1 desc
stepidx("..")
...
...
...
...
stepmult: just like the stepidx function, it takes as input the path to the execution-node where
the calculation of the multiplicity is based on. The same restriction as in the stepidx applies;the path must resolve to a step. It returns an integer, the multiplicity of the step. Examples:
...step 1 descstepmult(".")
...
...
...step 1 descstepmult(".")...
...
...
...
-
8/14/2019 Automated Scenario Test
16/27
...step 1 descstepmult("..")
...
...
...
...
As you can see from the examples above, we can pass a function as argument of another function.
This capability allows to build plethora of interesting combinations. The following grammar gives acomplete picture of the expression language of this test framework.
-
8/14/2019 Automated Scenario Test
17/27
stat: expr;expr: STR_LITERAL | INT_LITERAL | BIGDEC_LITERAL | NULL_EXPR | ANY_EXPR | func | bool_expr| mediator_expr | context_expr | rand_func | stepidx_func | stepmult_func | str_func;
bool_expr: BOOL_LITERAL | eq_expr | gt_expr | lt_expr | and_expr | or_expr | error_expr;eq_expr: 'eq' '(' expr ',' expr ')' -> ^('eq' expr expr);gt_expr: 'gt' '(' expr ',' expr ')' -> ^('gt' expr expr);lt_expr: 'lt' '(' expr ',' expr ')' -> ^('lt' expr expr);and_expr: 'and' '(' expr ',' expr (',' expr)* ')' -> ^('and' expr expr+);or_expr: 'or' '(' expr ',' expr (',' expr)* ')' -> ^('or' expr expr+);
mediator_expr: 'mediator' '(' STR_LITERAL ')' -> ^('mediator' STR_LITERAL);context_expr: 'context' '(' STR_LITERAL ',' STR_LITERAL ')' -> ^('context' STR_LITERALSTR_LITERAL);error_expr: 'error' '(' STR_LITERAL ')' -> ^('error' STR_LITERAL) | 'error' '(' STR_LITERAL',' STR_LITERAL ')' -> ^('error' STR_LITERAL STR_LITERAL);func: ref_func | list_func | range_func | concat_func | mult_func | sum_func | neg_func |mod_func | switch_func | div_func | int_func | wheel_func | short_func;str_func: 'str' '(' expr ')' -> ^('str' expr);range_func: 'range' '(' arith_func_param ',' arith_func_param ',' INT_LITERAL ')' ->^('range' arith_func_param arith_func_param INT_LITERAL);ref_func: 'ref' '(' VAR_NAME ')' -> ^('ref' VAR_NAME);
mult_func: 'mult' '(' arith_func_param ',' arith_func_param ')' -> ^('mult'arith_func_param arith_func_param);sum_func: 'sum' '(' arith_func_param (',' arith_func_param)+ ')' -> ^('sum'arith_func_param arith_func_param+);div_func: 'div' '(' arith_func_param ',' arith_func_param ',' INT_LITERAL ')' -> ^('div'arith_func_param arith_func_param INT_LITERAL);
mod_func: 'mod' '(' arith_func_param ',' arith_func_param ')' -> ^('mod' arith_func_paramarith_func_param);neg_func: 'neg' '(' neg_func_param ')' -> ^('neg' neg_func_param);neg_func_param: arith_func_param | bool_expr;int_func: 'int' '(' short_int_func_param ')' -> ^('int' short_int_func_param);short_func: 'short' '(' short_int_func_param ')' -> ^('short' short_int_func_param);short_int_func_param: arith_func_param | bool_expr;rand_func: 'rand' '(' arith_func_param ',' arith_func_param ')' -> ^('rand'arith_func_param arith_func_param);stepidx_func: 'stepidx' '(' STR_LITERAL ')' -> ^('stepidx' STR_LITERAL);stepmult_func: 'stepmult' '(' STR_LITERAL ')' -> ^('stepmult' STR_LITERAL);arith_func_param: INT_LITERAL | BIGDEC_LITERAL | ref_func | mult_func | sum_func | neg_func| list_func | range_func | div_func | int_func | mod_func | switch_func | wheel_func |rand_func | stepidx_func | stepmult_func;list_func: 'list' '(' map_entry_key_value (',' map_entry_key_value)+ ')' -> ^('list'map_entry_key_value map_entry_key_value+);wheel_func: 'wheel' '(' map_entry_key_value (',' map_entry_key_value)+ ')' -> ^('wheel'map_entry_key_value map_entry_key_value+);switch_func: 'switch' '(' switch_func_selector ',' '{' switch_func_case (','switch_func_case)* '}' ')' -> ^('switch' switch_func_case+ switch_func_selector);switch_func_selector: arith_func_param | concat_func;switch_func_case: map_entry_key_value ':' map_entry_key_value -> ^(SWITCH_CASEmap_entry_key_value map_entry_key_value);
map_entry_key_value: STR_LITERAL | NULL_EXPR | bool_expr | ANY_EXPR | concat_func |arith_func_param;concat_func: 'concat' '(' expr (',' expr)+ ')' -> ^('concat' expr expr+);STR_LITERAL: '"' ( options {greedy=false;} : . )* '"';INT_LITERAL: '-'? ('0'..'9')+;BIGDEC_LITERAL: '-'? ('0'..'9')+ '.' ('0'..'9')+;BOOL_LITERAL: 'false' | 'true';
NULL_EXPR: 'null';ANY_EXPR: 'any';VAR_NAME: ('A'..'Z' | 'a'..'z' | '_' | '$') ('A'..'Z' | 'a'..'z' | '0'..'9' | '_' | '$')*;WS: (' ' | '\t' | '\n' | '\r')+ {skip();};
Insert 5. The grammar of the expression language of the test framework
-
8/14/2019 Automated Scenario Test
18/27
Implementing action
Once the scenario is completely laid-out and received an approval, we are for the final step that is
implementing the actions that will actually carry out the tasks. There two ways of implementing action:(a) create the action class, or (b) script it directly inside the scenario XML.
Creating an action class
An action class is a (java) class that extends com.baktun.bstrd.director.Action. We
only have to implement one method: callExecute(...). The following insert contains the source
code of an example action class, PrintArgsAction that simply prints out all the arguments visible
from it.
package com.baktun.bstrd.unittest;
import java.util.Map;import com.baktun.bstrd.director.Action;import com.baktun.bstrd.director.ActionDefinition;import com.baktun.bstrd.director.Step;import com.baktun.bstrd.director.ValueProvider;
publicclass PrintArgsAction extends Action {public PrintArgsAction(ActionDefinition definition, Step step,
Map waitValPro,ValueProvider condValPro, ValueProvider expectValPro, Object mediator) {
super(definition, step, valPros, waitValPro, condValPro, expectValPro, mediator);}
public Object execute(Object resultFromPreviousAction) throws Exception {System.out.println(Thread.currentThread() + ":" + getCopyOfArguments());System.out.println("---");
returnnull;}
}
Insert 6. The source code of an example action class
The execute(...) method takes the result of the execution of theprevious action, that is the action
defined right before this action in the scenario XML (it's a way of sharing objects between actions,
beside sharing through context which will be explained in separate section). Consequently, theexecute(...) method returns an object (that will be passed on to the next action by the scenario
runner). If there's nothing to share, simply returns null.
From within the execute method, we can access the containing step, the mediator, and the arguments.
We can also store / retrieve values stored in the context. The following methods let you do theaforementioned things:
Step getStep()
T getMediator()
Object getArgument(String key)
Object getArgument(String key,boolean lookUp)
boolean hasArgument(String key)
boolean hasArgument(String key,boolean lookUp)
LinkedHashMapgetCopyOfArguments()
-
8/14/2019 Automated Scenario Test
19/27
LinkedHashMapgetCopyOfArguments(boolean lookUp) boolean hasValue(String key)
boolean hasValue(String key,boolean lookUp)
voidsetValue(String key, Object value)
Map getCopyOfValues()
Map getCopyOfValues(boolean lookUp)
voidunsetValue(String key)
voidunsetAllValues()
ExecutionNode getParent()
V getDefinition()
One thing worth mentioned here is the behavior of the method getArgument(...) and
getValue(...): they will throw an exception if the corresponding argument can not be found (in
the scenario XML) or there's no object associated with the specified key can be found (in the context),
respectively.
This is by design to force the SDET to state its intention in the scenario XML (in the case ofgetArgument(...)); if the value of a variable is intended to be null, don't leave it out in the XML.
Instead specify its value to null, like in the example below.
null
Insert 7. Specifying null in the scenario XML
Inside the action class, we handle it the following way:
...public Object execute(Object resultFromPreviousAction) throws Exception {
...Object theVarA = null;
if (getArgument("var_a") instanceof com.baktun.bstrd.director.Null) {theVarA = calculateDefaultValueOfVarA(...)
} else {theVarA = calculateValueOfVarABasedOn(getArgument("var_a"));
}...
}...
Insert 8. Handling null in the action class
Similarly if we want to state that var_a can take any value, set it to any in the scenario XML, as in the
example below.
-
8/14/2019 Automated Scenario Test
20/27
any
Insert 8. Specifying any in the scenario XML
Inside the action class, we handle it the following way:
...public Object execute(Object resultFromPreviousAction) throws Exception {
...Object theVarA = null;
if (getArgument("var_a") instanceof com.baktun.bstrd.director.Null) {theVarA = calculateDefaultValueOfVarA(...)
} elseif (getArgument("var_a") instance of com.baktun.bstrd.director.Any) {theVarA = calculateRandomValueOfVarA(...)
}else {theVarA = calculateValueOfVarABasedOn(getArgument("var_a"));
}...
}...
Insert 8. Handling any in the action class
To use that action class in the scenario XML, simply specify its fully-qualified name as the value of
class attribute of the appropriatenode. Example:
list("green", "orange")list(1980, 1990)
Insert 8. Using action class in the scenario XML
-
8/14/2019 Automated Scenario Test
21/27
Scripting the action
Alternatively, when the action is so simple15, we can directly script it inside the scenario XML. In that
case, there is no need to specify a value forclass attribute16in thenode. Instead, we have to
write directly those lines of code inside thenode (and don't forget to wrap guard it inside a
CDATA section). The scripting language is still java. The following insert contains an example of
scenario XML with scripted action.
action 1.2 desc
Random rand = null;if (action.getArgument("randSeed") instanceof Null) {rand = new Random();} else
{rand = new Random(action.getArgument("randSeed"));}
Integer theNextInt = new Integer(rand.nextInt());action.getStep().setValue("theNextInt", theNextInt);
System.out.println(Thread.currentThread() + ": Done! : " + theNextInt);return null;]]>list(10000, null)
Insert 9. Example of scenario XML with scripted action
From inside the script we can use the methods available in the Action class (page 19-20). However, we
have to prepend the invocation with action..
15 No exact definition of simple provided. It can be taken as: "no more than 5 lines of code", for example.
16 In fact, we must omit the class attribute.
-
8/14/2019 Automated Scenario Test
22/27
Passing objects through context
Other than passing the return of the execute method from one action to the next one (in the same step),
the framework allows actions to publish objects into the context, such that it can be later retrived andused by subsequent actions. The following code-snippet shows an example of how to use thesetValue and getValue methods for storing object in the context and retrieving object from the
context, respectively,
...public Object execute(Object resultFromPreviousAction)
throws ActionExecutionException {...RedBean redBean = processIt((GreenBean) getValue("theBeanThatIsShared"));setValue("theBeanToBeShared", redBean);...
}...
Insert 10. Using getValue and setValue
The above example, however, is not really useful because it tries to retrieve / store from the sameexecution-node; which translates to sharing object(s) only with itself, which is pretty much
meaningless. Normally we would retrieve / store from the execution node at least one level above. Thefollowing snippet shows how an example of storing an object in the step that the action belongs to, so
that subsequent actions within the same step can retrieve it. The snippet also shows an example of
retrieving an object from the step.
...public Object execute(Object resultFromPreviousAction)
throws ActionExecutionException {...RedBean redBean = processIt((GreenBean) getStep().getValue("theBeanThatIsShared"));
getStep().setValue("theBeanToBeShared", redBean);
...}
...
Insert 11. Store and retrive in / from a step
Finally, the following snippet shows how to store in and retrieve from the scenario.
...public Object execute(Object resultFromPreviousAction)
throws ActionExecutionException {...
RedBean redBean = processIt((GreenBean)getStep().getScenario().getValue("theBeanThatIsShared")); getStep().getScenario().setValue("theBeanToBeShared", redBean);
...}
...
Insert 12. Store and retrive in / from a scenario
-
8/14/2019 Automated Scenario Test
23/27
Nesting actions
Some scenarios requires nesting action(s) inside other action. We achieve nesting by, first, adding a
inside the. Further, inside the . Please note that unlike scenario, action can only have
one child step. The structure of a scenario XML (with nesting) can be depicted the following way:
Scenario Steps
Step
Actions
Action
Action
Step
Actions
Action
Action
...
Action
Step
Insert 13. Tree-structure of a scenario XML
Consider the following example which is taken from the real, current, project (LaCross). In the test
scenario we needed to setup 20 conferences. Each conference in turn has to have 5 segments, and eachsegment has to be filled with 100 participants.
In order to realize it, we have a step with multiplicity 20 step Create conference where we
define the BILLING_CONFIGID for each conference that will be created . Inside that step we havean action, a special kind of action that does nothing, com.baktun.bstrd.director.TransitAction
. It is a trick that allows us to go directly from a step to a (child)step without doing anything in
between (because in this case we don't really create the instance of conference before creating thesegments. The conference is only an abstract concept that is created at the time the segment is created).
We already saw a nesting (ofstep Create segment) inside an action (action Transit to segment
creation (and population)). Further down, we nest another step step Create participants
inside the Create segment action.
Another practical thing that can be observed in the example which has been described in the
argument's value resolution section is that we can use the argument defined in the parent node from a
child node, typically to construct a more complex pattern. For example: the value of
PARTICIPANT_ID defined in the Create participant action is based on the value of
SEGMENT_ID defined in the Create segment step. Similarly, the value of PHONE_NUMdefined in the Create participant action is constructed by concatenating the value of
BILLING_CONFID defined at the scenario level and the value of SEGMENT_ID defined in theCreate segment step.
-
8/14/2019 Automated Scenario Test
24/27
Scenario QA
eq(mediator("isStarted()"), true)
str(range(10101, 10120, 1)) concat("SpectelEmu_", ref(BILLING_CONFID))
short(sum(mult(stepidx("../.."), stepmult(".")),
stepidx("."), 1))
"OPERATOR_ASSISTED"falsenullnullnull
Map segmentMap = new HashMap();
segmentMap.put(SegmentProperty.BILLING_CONFID,
action.getArgument("BILLING_CONFID"));
segmentMap.put(SegmentProperty.CONF_NAME,action.getArgument("CONF_NAME"));segmentMap.put(SegmentProperty.CONFERENCE_ID,
action.getArgument("SEGMENT_ID"));if ((action.getArgument("IS_QA_ON") instanceof Null) == false) {segmentMap.put(SegmentProperty.IS_QA_ON,
action.getArgument("IS_QA_ON"));}if ((action.getArgument("IS_POLLING_ON") != Null) == false) {segmentMap.put(SegmentProperty.IS_POLLING_ON,
action.getArgument("IS_POLLING_ON"));}if ((action.getArgument("NRP_MODE") != Null) == false) {segmentMap.put(SegmentProperty.NRP_MODE,
Byte.valueOf(action.getArgument("NRP_MODE")));
}//System.out.println(segmentMap);mediator.getBridge().createSegment(segmentMap,
action.getArgument("NOTIFY_EVENTS"));return null;
]]>
rand(5, 10)
-
8/14/2019 Automated Scenario Test
25/27
short(sum(mult(sum(ref(SEGMENT_ID), -1),
stepmult("..")), stepidx(".."), 1)) falsefalseconcat("Participant_", ref(CONF_NAME), "_",
ref(SEGMENT_ID), "_", sum(stepidx(".."), 1))"Baktun"concat(ref(BILLING_CONFID), "-",
ref(SEGMENT_ID), "-", sum(stepidx(".."), 1))
Map legProperties = new HashMap();
legProperties.put(LineProperty.CONFERENCE_ID,
action.getArgument("SEGMENT_ID"));legProperties.put(LineProperty.PARTICIPANT_ID,
action.getArgument("PARTICIPANT_ID"));legProperties.put(LineProperty.CALL_TYPE, "DIAL-IN");legProperties.put(LineProperty.CHAN_TYPE,
Line.LineType.NET_USER_CHANTYPE);legProperties.put(LineProperty.IS_MUTED, true);legProperties.put(LineProperty.IS_IN_QA, false);legProperties.put(LineProperty.IS_IN_POLL, false);legProperties.put(LineProperty.PHONE_NUM,
action.getArgument("PHONE_NUM"));if ((action.getArgument("IS_MODERATOR") instanceof Null) == false)
{legProperties.put(LineProperty.IS_MODERATOR,
action.getArgument("IS_MODERATOR"));} else {legProperties.put(LineProperty.IS_MODERATOR, false);
}if((action.getArgument("NAME") instanceof Null) == false){legProperties.put(LineProperty.NAME, action.getArgument("NAME"));
} else {legProperties.put(LineProperty.NAME, "");
}if((action.getArgument("COMPANY") instanceof Null) == false){legProperties.put(LineProperty.COMPANY,
action.getArgument("COMPANY"));} else {legProperties.put(LineProperty.COMPANY, "");
}
mediator.getBridge().dialInParticipant(legProperties,action.getArgument("NOTIFY_EVENTS"));
]]>
action.getArgument("SEGMENT_ID") + " ===========================");mediator.getBridge().startQA(action.getArgument("SEGMENT_ID"));
]]>
-
8/14/2019 Automated Scenario Test
26/27
-
8/14/2019 Automated Scenario Test
27/27
Specifiying wait
In the above example we can seein action. Anode can be placed in any execution node
(scenario, step, or action). It will cause the execution of the node to be held on, until a specific
condition is met. The condition is the expression that we type inside thenode, which can be
either any function that returns integer, or any function that returns boolean.
In case the wait expression produces an integer (n), the scenario-runner will wait for (n) seconds beforeexecuting the node containing that wait. An example of this is the wait for action Stop QA section .
In case the wait expression is a boolean evaluation, as is the wait of scenario in the above example,the scenario-runner will first evaluate the wait expression. The execution moves on normally if the wait
expression evaluates to true. Otherwise, the thread in which the evaluation takes place will move
suspended state (i.e.: calling the wait() method), and only wakes up when either the specified number
of seconds (timeout attribute) has passed. When it wakes up, the wait expression will be re-evaluated.
Wash. Rinse. Repeat.
There's an advanced mechanism that causes the thread to wake up upon notification (thus avoiding the
polling). For that, we need to have an object an approriate type of object that is registered as alistener for the events that take place inside the mediator. Upon receiving the notification from the
mediator, that listener will have to invoke the notifyAll() method, such that the scenario-runner gets
back to live from the suspended state, and re-evaluates the wait expression. Additionaly, the scenario-
runner must be made aware of the existence of such listener. It is achieved by invoking thesetMediatorListener method from inside the very first action that the scenario executes.
The other important, but optional, property ofis timeout. When it's set to true, the scenario-
runner will evaluate thebefore it evaluates the(seespecifying condition below). By
default it is set to true.
Specifying condition
An execution-node can have anode, where we specify the condition under which the
execution-node will be executed. Thenode accepts boolean expression as its value. Normally
we check the value stored in the context or a property of the mediator against a specific value. See the
description of the context ormediator function to understand how to check the object stored in the
context or mediator.
Specifying expectation
Finally, an execution-node can have anode, where we specify the checkings to be performed
at the end of the execution of the execution-node. Just like the, it accepts any boolean
expression as its value, and normally we would make thecontext ormediator function together with
either the eq, lt, orgt function. Additionaly, only for the, we can also use the error
function, if we expect that the execution would throw an error.