using influence diagrams to understand testing workshop · isabel evans and stuart reid influence...

Isabel Evans and Stuart Reid Influence Diagrams Workshop June 2015

©Evans and Reid v6 final of 27

Using Influence Diagrams to

Understand Testing

Workshop

Stuart Reid and Isabel Evans

June 2015



Contents

1 Introduction: What is an influence diagram? ................................................................................. 4 1.1 Concept ................................................................................................................................... 4 1.2 Complexity .............................................................................................................................. 5 1.3 What’s different about an influence diagram? ....................................................................... 5

2 Using notation to show types of influence ..................................................................................... 7 2.1 Basic notation ......................................................................................................................... 7 2.2 Influence in the “same direction” ........................................................................................... 8 2.3 Influence in the opposite direction ......................................................................................... 8 2.4 Simplifying the notation .......................................................................................................... 8

3 Loops or cycles .............................................................................................................................. 10 3.1 Concept ................................................................................................................................. 10 3.2 Eating and feedback loops .................................................................................................... 10 3.3 Determining loop type .......................................................................................................... 12 3.4 Naming .................................................................................................................................. 12 3.5 Perception, reality and timing .............................................................................................. 13 3.6 A Testing loop ....................................................................................................................... 14

4 Black holes .................................................................................................................................... 15 4.1 Concept ................................................................................................................................. 15 4.2 Here in my car…. ................................................................................................................... 15 4.3 Black Hole – testing example ................................................................................................ 18

5 Catherine wheels .......................................................................................................................... 19 5.1 Concept ................................................................................................................................. 19 5.2 Exercise & Fitness ................................................................................................................. 19 5.3 Effect of time on the cycles ................................................................................................... 20 5.4 Testing example .................................................................................................................... 21

6 Realistic models (hybrids) ............................................................................................................. 22 6.1 Concept ................................................................................................................................. 22 6.2 Non-IT example ..................................................................................................................... 22 6.3 Testing examples................................................................................................................... 23



Table of figures

Figure 1 Simple example of an influence diagram .................................................................................. 4

Figure 2 Basic notation for an influence diagram ................................................................................... 7

Figure 3 Alternative basic notations for an influence diagram ............................................................... 7

Figure 4 Influence diagram showing driving speed affecting journey time............................................ 7

Figure 5 Influence diagram showing change “in the same direction” .................................................... 8

Figure 6 Influence diagram showing change “in the opposite direction” .............................................. 8

Figure 7 Simplified notation – if A increases then B increases and C decreases .................................... 8

Figure 8 If A increases then B will increase – green and red arrow notation ......................................... 9

Figure 9 Red and green arrows example ................................................................................................ 9

Figure 10 Simple loop – eating and hunger .......................................................................................... 10

Figure 11 Simple loop – eating and mood ............................................................................................ 10

Figure 12 Eating – closer to reality........................................................................................................ 11

Figure 13 Eating – overall the same, but with different naming .......................................................... 12

Figure 14 Showing a delay in time until an influence is felt ................................................................. 13

Figure 15 Training to close skills gaps – influences and outcomes ....................................................... 14

Figure 16 Simple black hole .................................................................................................................. 15

Figure 17 Balancing effect of perceived danger level on driving speed ............................................... 16

Figure 18 Black Hole of influences on braking distance ....................................................................... 16

Figure 19 A bigger Black Hole for car driving ........................................................................................ 17

Figure 20 Black Hole for Crowd Testing ................................................................................................ 18

Figure 21 Generic Catherine Wheel pattern ......................................................................................... 19

Figure 22 Initial influence diagram – exercise and fitness .................................................................... 19

Figure 23 Effects of exercise - start of an influence diagram - Catherine Wheel pattern .................... 20

Figure 24 Agile testers - a Catherine wheel influence diagram ............................................................ 21

Figure 25 More realistic influence diagram for the effect of exercise ................................................. 22

Figure 26 - Stability and counter-insurgency in Afghanistan ................................................................ 23

Figure 27 Dev/Test & QA Influence Diagram ........................................................................................ 24

Figure 28 Off-shoring & Education Influence Diagram ......................................................................... 25

Figure 29 Off-shoring Cost & Confidence Influence Diagram ............................................................... 26

Figure 30 Crowd Testing Influence Diagram ......................................................................................... 26

Figure 31 Fixing/Regression Testing Influence Diagram ....................................................................... 27



1 Introduction: What is an influence diagram?

1.1 Concept Very simply: An influence diagram is a pictorial way of showing why things have happened and what

might happen next. It is often used when we want to better understand a situation, especially when

we want help in deciding how to change that situation. So, it is an aid to decision-making,

particularly in complex situations with a lot of choices and uncertainties.

Definition from www.businessdictionary.com/definition/influence-diagram (24/4/2015):

Graphical aid to decision making under uncertainty, it depicts what is known or unknown at the time of

making a choice, and the degree of dependence or independence (influence) of each variable on other

variables and choices. It represents the cause-and-effect (causal) relationships of a phenomenon or

situation in a non-ambiguous manner, and helps in a shared understanding of the key issues.

Whether or not our (Isabel and Stuart’s) influence diagrams are unambiguous (as defined above) is

doubtful, but hopefully they would be if each of the elements were fully defined. As you will see,

this is not the case with all the influence diagrams presented in this text. However, they should at

least identify factors you should consider.

Definition from See: http://en.wikipedia.org/wiki/Influence_diagram (1-9-2014)

“An influence diagram (ID) (also called a relevance diagram, decision diagram or a decision network) is

a compact graphical and mathematical representation of a decision situation. It is a generalization of a

Bayesian network, in which not only probabilistic inference problems but also decision making problems

(following maximum expected utility criterion) can be modeled and solved.”

Figure 1 Simple example of an influence diagram

http://en.wikipedia.org/wiki/Influence_diagram



1.2 Complexity If we look at the example above we know that more things influence whether we will get a parking

space than the amount of sunshine (and is this forecast sunshine or what we can see out of the

window now?). We also know that many factors affect the number of visiting tourists.

However, we will see more realistic, and so more complex influence diagrams later. As it stands the

diagram is correct – influence diagrams do not have to include everything – just what we are

interested in.

As we work on influence diagrams we often (normally!) find they are hard to do. This means that we

might not fully understand the situation, and recognizing that is, in itself, useful.

We should expect that our influence diagrams help us model aspects of real life, but do not

completely model it. Remember, no model is complete. Real life is complex.

1.3 What’s different about an influence diagram?

1.3.1 Certainty vs Probability

As software testers, you will most likely be aware of cause-effect diagrams, which (along with their

corresponding decision tables) have been used by testers for several decades as an aid to test case

generation. We also make use of state transition models, use case diagrams and textual

specifications when we are trying to understand required system behaviour and to design test cases.

The important word here is ‘specification’. These all provide models of the required behaviour of a

system. We all know that these specifications are rarely complete, often contain mistakes and may

well define behaviour that although it reflects what the customer requested is not what the

customer actually needs. Nothing’s perfect. But the point about these models is that they are trying

to define (to varying degrees of success) what the system should do. As specifications they

represent a high degree of certainty.

For instance, imagine our specification contains the statement:

“If Bank-account-balance is less than £0.00 then charge interest of 5%.”

We know with absolute certainty that if the bank account is overdrawn (balance less than zero) then

interest of 5% should be charged. No ifs and no buts.

In contrast, an influence diagram works in the realm of probabilities; in an influence diagram the

links between the nodes do not show certainties but possibilities. This means they are not

appropriate as specifications, but are far better at modelling the range of possible behaviours of a

system. Note that we are using the term ‘system’ here in its widest sense and include human

behaviour.

They allow us to identify what factors are important about a situation. These factors can be within

the system or in the system’s environment (we are deliberately loose with our definition of the

system boundary). These factors can be anything that affects (influences) another part of the

system, and so can include the system parts, people and other systems. In fact anything that helps

us understand how the system ‘ticks’. One of the great benefits of using influence diagrams is that



they allow us to identify factors that we would otherwise overlook. Once we have identified what

influence what, we can then analyse the situation by asking ourselves “What if?” questions to see

how, if at all, we can make the situation better.

Both techniques are useful, they complement each other and used together widen the scope of

what we consider when testing.

Not just the IT system but also the business processes.

Not just the business process but also customer touch points.

Not just the digital and certain but the analogue, human and uncertain.



2 Using notation to show types of influence

2.1 Basic notation Influence diagrams come in a wide variety of notations with varying levels of detail. We have

decided to try and keep our influence diagrams very simple and have included only the most basic

concepts. If you really get into using influence diagrams and perhaps decide you want a computer

simulation of your system then you will probably find you need to include some of the more

sophisticated concepts (and even use tools!). However, it’s probably best to start simple…

A basic influence diagram consists of nodes and arrows. In the diagram below, at the simplest level

we can see that A influences B.

Figure 2 Basic notation for an influence diagram

There is nothing special about the fact that we have used an oval above for the nodes, nor anything

special about the font or arrow style. See below for alternative notation meaning exactly the same.

Just use whatever is easiest for you that still gets the message across.

Figure 3 Alternative basic notations for an influence diagram

Here is a simple example:

Figure 4 Influence diagram showing driving speed affecting journey time

The speed we drive affects how long our journey takes.

Influences can work in a couple of ways; an influencing factor might cause either an increase or a

decrease in the influenced factor.



2.2 Influence in the “same direction”

Figure 5 Influence diagram showing change “in the same direction”

In this diagram, when A increases that causes B to increase. This is shown by the pluses (+) on the

arrow. Simple. However, there is also a second interpretation: When A decreases that causes B to

decrease. So the plus signs mean that they both increase or they both decrease, together.

For instance if A represents ‘bank balance’ and B represents ‘happiness’ then a positive change in

bank balance (A) results in a positive change to happiness (B) – and if the bank balance goes down

then happiness decreases.

2.3 Influence in the opposite direction

Figure 6 Influence diagram showing change “in the opposite direction”

In this diagram, the influencing works the other way around; when A increases, that causes C to

decrease (and when A decreases that causes C to increase). This is shown by the plus (+) and minus

(-) on the arrow. When a change occurs the movement is in the opposite direction.

Here, if we leave A as representing ‘bank balance’, then we can imagine that C could represent

‘money worries’.

2.4 Simplifying the notation

Figure 7 Simplified notation – if A increases then B increases and C decreases



In practice we do not need the + at the start of the arrow – it is “always there” so we do not draw it.

This diagram tells us that if A increases, then B will increase and C will decrease. Alternatively, if A

decreases, then B will decrease, but C will increase.

The + and – are known as the polarity of the influence arrow.

In today’s world of colour screens and printers, it can be easier to use colour coding on the influence

arrow rather than the + and -. We’ll use green for a positive polarity and red for a negative polarity,

which means that figures 7 and 8 are both presenting the same model.

Figure 8 If A increases then B will increase – green and red arrow notation

Figure 9 Red and green arrows example



3 Loops or cycles

3.1 Concept Sometimes the influences move in a loop or a cycle, which makes the influence diagram much more

interesting. Let’s look at some examples…

3.2 Eating and feedback loops The examples used here are deliberately both simple and simplistic (“treating complex issues and

problems as if they were much simpler than they really are”).

Isabel says, “If I am hungry then I eat, then I become full, and do not want to eat so much.” As you

can see, it doesn’t matter where we start in the loop, the influence diagram still works.

Figure 10 Simple loop – eating and hunger

Stuart says, “If I eat, then I put on weight, and I feel sad about myself. If I feel sad about myself I eat

to comfort me. If I eat, then I get fatter…” Again, it doesn’t matter where we start in the loop, the

influence diagram still works.

Figure 11 Simple loop – eating and mood



There is a subtle, but important difference between these two loops.

If we consider the eat-fullness-hunger loop in figure 10, we can see that Isabel has got a nice

‘balanced’ approach to her diet. She eats until she becomes full, her hunger abates and so she stops

eating. Depending on how full and how hungry she gets then all should be well (the actual values

are outside the scope of the influence diagram – it simply tells us what the influences are, not at

what value they should be set). This is an example of a ‘balancing’ loop.

Stuart, on the other hand, has a real problem. If this influence diagram were complete (i.e. telling us

the whole story of Stuart’s diet) then he is going to explode! Where we have loops that would keep

the behaviour increasing more and more (or decreasing more and more) with no influences to stop

the cycle then we have what is known as a reinforcing loop. Happily, Stuart has not exploded yet, so

we know that the eat-fat-sad influence diagram is not a complete model of his dietary behaviour.

In practice reinforcing loops cannot exist in isolation, they need something to allow them to be

controlled – or, more than likely, there will be a disaster. Think back to steam engines before they

had limiters fitted to them and the frequent accidents that led to some of the first regulatory

engineering standards. A pathological cycle is a one where there is no balancing mechanism, and the

situation becomes untenable: dangerous, life threatening, destructive.

If we join the two influence diagrams together, the result becomes closer to reality, but, naturally,

begins to make the diagram more complex as we try to model both behaviours. With both loops we

can see that there are now two influences on eating. We are influenced to eat when we are sad, but

there is a balancing influence that if we are not hungry we are influenced not to eat. If we

(simplistically) ignore all other influencing factors (that we have not included in the diagram) then

the relative power of these two influences will decide how much we eat and so how fat we are.

Figure 12 Eating – closer to reality

Note that the loops in the influence diagram have been labelled to make it obvious which is the

balancing and which is the reinforcing loops.

For this influence diagram, the “eat” node is obviously central to how the system behaves. When we

want to encourage a particular behaviour, we look for these nodes that intersect between balancing

and a reinforcing cycles, and try to identify how we might best influence this factor.



It is worth noting that the reinforcing eat-fat-sad cycle we used above with all positive polarity

influences only applies to a subset of the population (but it may be a large subset given current

levels of obesity). For some people when they get sad they stop eating (or eat less), in which case

for them the influencing arrow from sad to eat would need to change to a red (negative polarity)

arrow.

3.3 Determining loop type As we saw above, we can use simple logic to determine if straightforward loops are balancing or

reinforcing. However, as our influence diagrams get more complex determining the type of loop can

become more difficult. Whether a loop is balancing or reinforcing can be easily determined by

following the link polarity around the loop and observing the following rules:

• Two successive links with polarity + and + are equivalent to one link with polarity +

• Two successive links with polarity + and − are equivalent to one link with polarity −

• Two successive links with polarity − and + are equivalent to a link with polarity −

• Two successive links with polarity − and − are equivalent to a link with polarity +

Once you get back to your starting point you are either left with a + or a −. In the former case, you

have found a reinforcing loop, in the latter case a balancing loop.

3.4 Naming You will probably have noticed that how the factors are named can have a big impact on the

influence diagram. In the influence diagram below, the ‘sad’ and ‘fullness’ factors have been

replaced by ‘happy’ and ‘emptiness’ factors. Sometimes how you name factors can be a deliberate

choice to get a certain message across to the viewer. By changing these names and so reversing the

meaning of these two factors, you will see that it was necessary to reverse the polarity of the

influencing arrows into and out of these factors.

Figure 13 Eating – overall the same, but with different naming

Of course, simply changing the names does not change the polarity of the loops – they are still the

same as before. Try using the rules to check.



3.5 Perception, reality and timing

Figure 14 Showing a delay in time until an influence is felt

With some influences, there is a delay between something actually happening and the perception of

it happening. For example, if a quality improvement is made, or an improvement in testing, there is

often a delay between the improvement taking place, and stakeholders acknowledging or realising

the improvement has happened. We can show this by marking the influence arrow with a double

diagonal line.

There are a number of situations where it is important to acknowledge and document actual versus

perceived states. One example is in the driving influence diagram earlier where there is difference

between perceived and actual risks on the road, especially after alcohol consumption. Some

examples where there is a difference or a delay include:

– Perceived risk v actual risk;

– Perceived quality v actual quality;

– Perceived improvement v actual improvement;

– Perceived cost/benefit v actual cost/benefit.



3.6 A Testing loop Notice that this influence diagram follows the same pattern as the one above. There is a balancing

loop and a reinforcing loop.

Figure 15 Training to close skills gaps – influences and outcomes

The balancing loop says: “if the skills level in a team drops, then the gap between what the

organisation needs and the team’s skills may increase. If the skills gap increases then the need for

training may increase. If the training is increased, the skill level should go up, and the skills gap

should decrease.”

The reinforcing loop says: “If training goes up, the attainment of certifications may go up. If the

attainment of certifications goes up, then the employability of the team members may go up. If

employability increases, an individual may decide to pursue further training based on their past

experience to make themselves even more employable.”

As we noted with the eat-fat-sad loop before, the influence diagram will normally only apply to a

subset of the whole population. Here again, with the training-certification-employability loop there

will be a different population where the polarity of the arrow from employability to training would

be negative (i.e. they stop training once their employability means they have got the job they

wanted).



4 Black holes

4.1 Concept When we create non-trivial influence diagrams we often identify a node that we can see has a major

influence on how the system works. In the influence diagram below nodes A, B and C are in a

reinforcing loop and we can imagine that node B has been identified as the best place to try and

control this loop. In such a situation we first need to identify influencing factors for node B. In the

diagram you can see that we have identified five influencing factors (nodes D, E, F, G and H). Next

we must decide which of these can be used to satisfy our goal. Some will be outside our control, but

we may be able to exert an influence over the others. And, of course, we may need to add further

detail by identifying those factors that influence these factor, etc. Note that although we have used

the example of attempting to control a reinforcing loop here, the concept of identifying an

important node, and then identifying factors that influence it, may be used for a number of different

situations.

When we add such influencing factors to an influence diagram, we often end up with a node (the

one we are trying to change) with a number of arrows pointing at it (as can be seen in the figure

below). As we add more influencing factors it begins to look as if the influenced node is sucking in

the influencing factors like a black hole; hence the black hole pattern.

Figure 16 Simple black hole

4.2 Here in my car…. When we drive, we are typically balancing a number of risks (e.g. accident, speeding ticket) with a

number of goals (e.g. get to destination, in a reasonable time). Here is a simple balancing loop



showing a relationship between the speed we are driving, our braking distance, the actual accident

risk, and our perception of the danger we are in.

Figure 17 Balancing effect of perceived danger level on driving speed

There are numerous other influences on this loop of driving speed, perceived danger, accident risk

and braking distance including alcohol consumption, weather conditions and road condition.

Imagine if we decide to concentrate on those factors influencing braking distance. If we map these

factors onto the influence diagram we see that we soon get a ‘black hole’ effect with all the

influences going in towards the centre.

Figure 18 Black Hole of influences on braking distance



In practice, we would need to also consider the influences in more depth, and also not simply

concentrate on braking distance.

Figure 19 A bigger Black Hole for car driving



4.3 Black Hole – testing example As an example of a ‘black hole’ influence diagram for testing see below an influence diagram created

to identify factors that influence the choice of using crowdtesting as opposed to testing in-house.

Our node of interest is ‘CROWD DIFFERENTIAL’, which represents the cost/benefit difference

between crowdtesting and in-house testing, where a positive crowd differential means that crowd

testing should be the preferred option.

Figure 20 Black Hole for Crowd Testing



5 Catherine wheels

5.1 Concept The Catherine Wheel pattern could be thought of as the opposite (or complementary) pattern to the

black hole. With the Catherine Wheel pattern we attempt to identify nodes that are influenced by

our original core node (or nodes). A typical use of such an influence diagram is where we are trying

to identify how things are affected by our system. Marketing and change management are examples

of such a use, where we are looking for all the different stakeholders who are affected by a change

to a system (or the creation of a new system).

Figure 21 Generic Catherine Wheel pattern

5.2 Exercise & Fitness Let’s consider a requirement to sell the benefits of exercise. We could initially create an influence

diagram as shown below. Note that this is pathological reinforcing loop, but as we are mainly

interested in using it as a marketing tool we probably don’t need to worry about the dangers of

over-exercise at this point.

Figure 22 Initial influence diagram – exercise and fitness



We now need to identify how the factors in our reinforcing loop can cause other beneficial effects

that we could use to ‘sell’ exercise. Using the Catherine Wheel pattern we are looking for positive

factors that spin-off from our initial loop. At the next stage we may well arrive at the influence

diagram below:

Figure 23 Effects of exercise - start of an influence diagram - Catherine Wheel pattern

5.3 Effect of time on the cycles Our influence diagrams provide no additional notation for modelling the notion of time. There is

some implicit duration in each of the loops, but nothing explicit. The cycles within influence

diagrams may well be affected by the passing of time. In the above influence diagram it is clear that

the ‘exercise-endorphins-feel good’ loop works on a far shorter cycle than the ‘exercise-fitness-body

image- feel good’ loop; one is in the region of minutes, while the other is probably weeks. If we

were to make this model more realistic then we would need to add some negative factors (if only to

control the reinforcing loops), one of which would be ‘injuries’ caused by exercise. If this were

added, we could see another loop that works at a far longer timescale still, where, for instance, too

much exercise while younger can cause injuries (e.g. arthritic knees) that only become evident

decades later.

Our earlier influence diagrams on eating and diet provide another good example of where different

loops work over different time periods, as food can give instant gratification, but dieting is a longer

term cycle.

In the same way it is easy to model testing projects, patterns and behaviours over-simplistically.

Finding bugs is good, but eventually needs to stop to allow other factors into consideration, such as

meeting a delivery deadline.



5.4 Testing example

Figure 24 Agile testers - a Catherine wheel influence diagram

In this influence diagram we show how the inclusion of three different types of testers (developer-

testers, testers integrated into the agile team, and specialist testers external to the agile team)

affects an agile project.

Notice that this is a very simple diagram. We’ll see more complex and realistic examples in the next

section.



6 Realistic models (hybrids)

6.1 Concept Once we start to try and model more influences we get a more complex diagram, which is a hybrid

of all the patterns we have considered so far. This section provides examples of increasing

complexity. A realistic / hybrid model is one where the complexity of the modelling is closer to

reality, where we will find multiple influences into and out of each node, multiple loops, and both

black holes and Catherine wheels.

6.2 Non-IT example As we have already mentioned, our influence diagram for exercise is far too simplistic – at its core is

a reinforcing loop with no incoming influences to control it. Left as it is the model suggests that the

individual would exercise more and more, getting fitter and fitter, feeling fantastic about

themselves. In reality we cannot exercise to the exception of everything else (exercise does not only

have positive effects – think of the injuries caused by exercise we mentioned earlier). To exercise we

must also eat. So perhaps we should combine our eating and exercise influence diagrams for a more

complete picture. We also have work (exercise and food are rarely cost-free), family and other

leisure activities to take into consideration. The usual problem is deciding where to draw the

boundary of the influence diagram.

Figure 25 More realistic influence diagram for the effect of exercise



In this example, we can see for example that the amount of exercise we do influences fitness,

endorphin levels, but also injuries and pain. The more exercise we do, the more likely we are to get

an injury. Additionally, the more exercise we do, the more we spend on sports gear, reducing the

money we have to spend on other things (and we also have less time for other things such as family,

work and other social activities). In this diagram the loops are not explicitly identified, but you can

see that there are multiple loops of various complexity presented in the diagram.

See below for an example of a far more complex influence diagram (showing a slightly different

notation with some notion of time delay) modelling the stability and counter-insurgency in

Afghanistan post 9/11 produced by PA Consulting.

Figure 26 - Stability and counter-insurgency in Afghanistan

6.3 Testing examples In this section we provide a number of influence diagrams that illustrate the effect of testing

approaches, models and resourcing.



Figure 27 Dev/Test & QA Influence Diagram

In the above diagram there are two groups working on delivering systems; the dev/test group (who

are responsible for development and initial testing) and the QA group, who are responsible for

acceptance testing. The three ‘extra xxx budget’ factors are all options used when customer

satisfaction decreases. Note that if we just add budget for QA testing (late in the life cycle) then we

run the risk of making things worse by finding more defects thus reducing time for developing new

features with reasonable quality.



Figure 28 Off-shoring & Education Influence Diagram

The above diagram tries to show the benefits and dangers of off-shoring. You can see that there is a

simple balancing feedback loop that shows that as off-shoring becomes more popular its costs

increase and the onshore/offshore differential decreases. If the demand for onshore work

decreases then onshore staff availability increases and salaries decrease in a classic supply and

demand cycle. In the longer term if onshore salaries are low then demand for university courses

drops and we eventually end up with a lack of availability of onshore staff, which should then

increase salaries. However, if this lack of availability is not managed then this will increase demand

for off-shoring, potentially reducing the availability onshore to an extent that the universities cannot

meet future demands.

The following influence diagram considers a different aspect of offshoring, by looking at the costs

involved (over and above those for in-house testing) and what influences confidence in the testing

done offshore.



Figure 29 Off-shoring Cost & Confidence Influence Diagram

Figure 30 Crowd Testing Influence Diagram

Earlier we looked at the market forces that influence the decision to use crowd testing. Here we

consider crowd testing from the perspective of the crowd tester. Project invitations are received by



crowd testers with the right skills registered, language and test environment (e.g. specific tablet

model and version), who also have a sufficiently high rating with the crowdtesting organization. If

the crowd tester has availability and the appetite then they volunteer, and, if they have been quick

enough (there are limited ‘spaces’ available on each project) then they are assigned to the project.

As you can see the crowd testers are only paid for unique, properly-documented defects, so speed is

important to the crowd tester.

Figure 31 Fixing/Regression Testing Influence Diagram

In the above diagram, if there are time pressures on the project then less defects are found, and so

less fixing and retesting/regression testing is required, so easing the time pressure (two balancing

loops). In isolation, this seems OK, however time pressure results in less test coverage, so reducing

confidence, which, in turn, results in a demand for extra testing (a reinforcing loop). Similarly, time

pressure leads to features not being developed, which results in a demand for extra development of

these features, so resulting in more time pressure (another reinforcing loop). With this as a start,

consider what questions need to asked and other factors added. For instance, what happens if we

reduce the required test coverage level? And although finding less defects reduces time pressure,

what effect does it have on the delivered quality?

using influence diagrams to understand testing workshop · isabel evans and stuart reid influence...

Documents