whitepaper agile rrbt+tf

Whitepaper Combining Agile, RRBT & TestFrame

About LogicaCMG in testing

LogicaCMG is a major international force in IT services and a thought leader in testing methodologies. It employs around 7,000 testers out of total workforce of 40,000 across the globe, has test factories in three different continents, and has developed industry-leading testing methodologies and techniques from millions of hours testing over 30 years. LogicaCMG's focus is on helping leading organisations worldwide to increase the predictability of IT changes by a business outcome-led approach to testing while improving quality, reducing time to market and lowering their IT costs. The company provides managed testing and managed test environment services, business acceptance management, quality assurance and niche testing services across diverse markets including telecoms and media, financial services, energy and utilities, industry, distribution and transport and the public sector. Headquartered in Europe, LogicaCMG is listed on both the London Stock Exchange and Euronext (Amsterdam) (LSE:LOG; Euronext:LOG) and traded on the Xternal List of the Nordic Exchange in Stockholm. More information is available at www.logicacmg.com.

Combining Agile, RRBT & TestFrame

Author: Peter Kalmijn

Contents 1 WHAT IS AGILE? 0

2 AGILE, RRBT AND TESTFRAME 2

3 MAPPING AGILE AND RRBT 3

4 MAPPING AGILE AND TESTFRAME 5

5 AGILE TEST WARE 8

6 RRBT, TESTFRAME AND AGILE LIFECYCLES 10

7 CONCLUSION 14

A GLOSSARY OF TERMS 15

B RECOMMENDED READING 16

© Peter Kalmijn, LogicaCMG, 2007. All rights reserved

00

1 What is Agile? “Agile is an iterative and incremental (evolutionary) approach to software development which is performed in a highly collaborative manner by self-organizing teams with "just enough" ceremony that produces high quality software in a cost effective and timely manner which meets the changing needs of its stakeholders.” definition by Scott W. Ambler, one of the leading thinkers of the Agile movement.

This paper will demonstrate ways RRBT TestManagement and TestFrame can be used inside Agile environments and contribute strongly to the success of Agile projects. Also it will show how TestFrame can be applied in an Agile way itself.

Business sponsors, Project Managers and Process Improvement initiatives continue to search for means to improve predictability of Software development projects many of which overrun in terms of time to market and/or budget. In response to such overruns, quality or functionality is usually sacrificed.

Various methodologies have evolved to solve this dilemma. Of these, Agile emerged in the late 1990’s. The roots of Agile originated from the insight that a common contributor to many project failures are poor people management and communication. Individuals who had separately solved this issue in individual projects coalesced as the Agile Alliance and as result in 2001 the Agile Manifesto was published:

Agile Software Development Manifesto

Individuals and interactions over processes and tools Working software over comprehensive documentation

Customer collaboration over contract negotiation Responding to change over following a plan

www.agilemanifesto.org

Key features are: • close collaboration between project managers, developers, testers and business subject

matter experts • emphasis on face to face communication and agreement over written documentation • frequent delivery of software with demonstrable new deployable business value • tight, self-organizing teams rather than on rigid plans and mandated deliverables • ways to manage the inevitable change to requirements without the need to to evoke a

crisis

Do Agile methods replace conservative or plan-oriented methodologies such as for example Prince2? The answer is No; they can successfully co-exist. Wise project managers already look to combine best practice from the available methodologies. Cit:1 ‘’.. the way PRINCE is applied to each project will vary considerably, and tailoring the method to suit the circumstances of a particular project is critical to its successful use”. Agile methods do include practices that can be applied generally - regardless of methodology.

1 PRINCE2 par. 2.3: PRINCE in context

© Peter Kalmijn, LogicaCMG, 2007. All rights reserved.

01 Combining Agile, RRBT & TestFrame

Appropriately applied, benefits of Agile include • Faster software deliverables that are Business Value orientated, focussed on what is

needed by the business right now. • Improved analysis and management of Risk (Project Risks and Product Risks) • Frequent feedback to and re-direction from business facilitating better and faster decision

–making and so delivering more value o Lower cost of change o Early measurable business benefits

• Improving communications within and between project team & customer Agile and Testing In the Agile environment, with its de-emphasis on process, there are many challenges for the Tester. What is the role of the tester in Agile development projects? How does Testing fit within Agile lifecycles?

In seeking to answer these questions, this paper will focus on: 1) Applying TestFrame in Agile ways, responding to change 2) Using RRBT and TestFrame inside Agile projects (SCRUM, XP)

For this we look into: • the Role and Practice of Testing inside Agile projects • The practical application of RRBT TestManagement and the TestFrame Framework

within Agile projects • Agile Test Ware documentation practices

One of the controversial things about testing the agile way is that testing is far more explicitly a service role rather than the more traditional independent risk and assurance role. The ability to facilitate progress in the right direction is what counts. The tester serves the business expert and the developers to this end. The general purpose of testing remains valid: to provide timely information about the qualities of the system being build. Testing is used particularly in agile projects to boost agility of both the design and development process:

Testing is structurally interwoven with Requirements engineering, Software Development, and Analysis and Design.

Testing actively engages in defining requirements – Agile projects use testing to clarify requirements and design.

Completion is only reached after passed tests – testing has become the real measure of progress.

Continuous build and test – regression testing captures faults at the earliest stage possible, while it is still quick and cheap to fix. This practice also supports safe refactoring. Use of found flaws to tighten tests – future tests will capture this type of flaw. Next we will investigate how RRBT and TestFrame can be applied in an agile way and how they are blending well into agile development methods.

Testing is an Agility enabling factor

Agile and Testing


02

2 Agile, RRBT and TestFrame It is important to use a methodology that really fits the organisational situation. Both RRBT and TestFrame are designed to enable this important fit to happen. Before continuing there is a need to ensure a clear understanding of both RRBT and TestFrame.

Risk & Requirement Based Testing (RRBT) is a very successful TestManagement method that has several strategic advantages: • Informed decisions about management of risk; where to reduce risk by investing in testing

and were to accept risk and so save time and test budget • Early detection of vague or missing requirements by mandating (peer) reviewing of

requirements for testability and linking risks with requirements • Informed decisions about priority of testing; focusing test effort on areas of highest risk

first • Communicating risk and associated decisions required in the language of the stakeholders

The core of the RRBT test strategy is risk management through appropriate application of test effort, which is carried over to all RRBT Test Management activities and to TestFrame test execution. This is described in more detail in “Successful Test Management: an integral approach” [Pinkster 2004].

TestFrame provides a testing framework within which RRBT test strategy, test set up and test tools are tuned to complement one another, supporting the entire testing process, from preparation to completion. TestFrame has been for years extensively by scores of organizations and in many projects all over the globe. The TestFrame vision is that all testing material should be simple to maintain and should be reusable. The core components of this vision are: fitting, structuring and tooling. Fitting - the testing process needs to fit the organization and its specific ways of systems development. Structuring - executed tests will only offer quality control if they are set up in a structured manner and are easy to maintain. Tooling - aid of automated tools is indispensable in a modern testing process. TestFrame is an open standard, is supported by several tools which are provided in the TestFrame support box and is described in “Integrated Test Design and Automation” [Buwalda 2001].

The agile testing movement has gained increasing popularity within development and test departments because of its appeal of the perceived ease of setting up “lightweight” processes vs. traditional “heavyweight” processes. Both operational culture and organization must be considered too. In the implementation of Agile, as in any other process change, RRBT TestManagement and TestFrame are readily adaptable to any organisation and process frameworks. Therefore, a healthy and effective combination of RRBT TestManagement, TestFrame framework and Agile test practices is an appealing option; with Agile test practices oiling the process and RRBT and TestFrame contributing to support sustainable, repeatable and improving agility over time.

Adopting ‘Agile testing’ inside more traditional method strongholds may prove tricky, but here are some practical first steps: Plan for training, select the right pilot project, and staff with the ‘A’ team. One agile practice or technique at a time – people need time to get used to change. Focus on selling the benefits and demonstrating the returns - While certain agile methods may not fit in particular projects, some of their practices or techniques do provide valuable benefits when applied in certain contexts. You will win people over by producing measurable results.

What is RRBT

What is TestFrame

Combining with Agile

Agile adoption



3 Mapping Agile and RRBT In this section we look into the mapping of RRBT TestManagement and Agile. Contrary to Agile, traditional development approaches try to ensure the projects success by rigidly following strict processes and procedures. This may be good for repeatedly assembling the same product, but rarely works for the innovative and fast creation process of software systems delivering business value recognised by the users. Agile seeks to do this, and RRBT TestManagement complements Agile by tracking the evolving business needs and safe guarding their implementation into the system build. Differences Agile Conservative, plan-oriented Outcome result guarantee process guarantee, repeatable (Test) Project leader

guiding vision, communicating Plan driven, taskmaster

Plan Responding to change Following the Plan Processes Simple rules & open information Defined work processes and controls Teams Organic teams Fixed roles Stakeholders Active Stakeholder participation upfront involvement Design Evolving according to Stakeholder

Value Big upfront design

Delivery Short increments and frequent delivery in UAT and BAT to real users

Delivery on the end of the project

RRBT Risk Orientation – Complimentary to the Agile Project Management principle of mitigating Product Risks (mostly architectural) at an early stage.

Product Risk Assessment & Classification – By classifying according to ISO9126 standard RRBT provides a sound basis for an Agile Test Strategy.

Combining Product Risks and Requirements to determine Priorities – This fundamental RRBT practice provides Agile Project Management with an excellent tool to manage risks and to deliver the most valuable requirements first.

Risk Visibility – Since key Stakeholders are actively involved it is much easier to synthesize both risks and requirements to reach a cohesive baseline. RRBT emphasizes the need for quality, face-to-face interaction with Stakeholders to identify and prioritize risk. RRBT supports continuous reporting to individual stakeholders, linking individual risk and the associated mitigations to the stakeholders concerned.

Traceability – Both product Requirements and product Risks need to be traceable to the Test Cases and Test Conditions that validate them. Getting this right is one of the greatest challenges within an Agile project. RRBT supports this by supplying practical guidelines.

RRBT Cluster Cards – matches closely with SCRUM Feature Groups.

SCRUM Sprints (iterations) – are supported by RRBT/TestFrame Strategic Test Slicing Method (STSM)

Active Stakeholder Participation – Since key Stakeholders are actively involved it is much easier to synthesize both risks and requirements to reach a cohesive vision as well as a cohesive set of requirements and risks to base testing on.

User story –creation and refinement is supported through workshops.

Acceptance tests – are supported by involving customer subject matter experts to create Test Cases and their variants. Once automated the Test Cases are run frequently for regression monitoring purposes.

RRBT practices supporting Agile

Agile practices supporting RRBT


04 Refactoring and optimization - is supported by tactically using TestFrame Automation once functionality has been accepted.

Continuous Testing – This finds defects in the earlier stages of software development, minimizing their effect on the time schedule. TestFrame supports this practice by making Test Automation easier to implement.

Feature Driven Development (FDD) – A group of features usually can be mapped directly to a TestFrame test cluster. Combined with Feature-Based Design, Planning, Engineering, Testing and Reporting; FDD enables the whole project to be geared around features and associated Test Clusters. TestFrame has as a standard feature for Product Quality Monitoring and Reporting on Test Clusters (i.e. Feature Groups).

Test Driven Development (TDD) – enables early involvement, so getting more grip on Product Quality upfront.

Joint Test Ware Design (JTD) – Supports collaborative development

Concurrent Design, Engineering and Testing– This is especially useful in projects evolving in response to external changes.

Agile Management Techniques – many of these are not new – many have been around for a long time, and represent best practices of “traditional” project management. Using them within the right context, they strengthen the overall approach.

Agile Meetings – Agile requires short and frequent team meetings and meetings with stakeholders. These include Stand up Meetings, Scrum meetings, Whiteboard Sessions and the like. This level of communication with business and development facilitates the communication of risk that is a key requirement of RRBT.

In summary, the traditional test methodologies are sometimes considered bureaucratic, prescriptive or "predictive" in nature. The emphasis is that the process is followed. Lightweight and Agile methodologies on the other hand set the focus on the outcome and are much more flexible about the process that leads to it.

Implementing Agile practices requires both business insight and leadership skills. What method to implement depends on the context of the project, which might include principles, problem type, team dynamics and of course organizational culture and structure. The right choice between using a light or heavy methodology will significantly influence the success of the project. Keep in mind: Managing different stages and parts of the RRBT agile managed project could well demand a different balance between a light scaled version of TestFrame and a heavy scaled version of TestFrame. Both the balance and the flexible scalability determine success. The right balance is what makes a project really agile.

Agility balance indicators Aspect Rationale Team size Greater number of staff needs more management overhead. Project criticality Both urgency and critically of the project greatly influence the balance. Technology used The usage of groupware or other aids to communicate quickly and at

any time. Documentation More rigid methodology needs more documentation. Training Effective training to key support staff and project managers is required. Best practices/lessons learned

Past lessons learned and good practices should be analyzed.

Examination of existing processes

The maturity of existing processes will influence the pace at which a test project will progress.

Test tooling Right tooling can greatly contribute to the agility of the team.

Scale for Agility



4 Mapping Agile and TestFrame In this section of the paper we will investigate ways to combine TestFrame and Agile methods and practices. TestFrame is a powerful tool to support common Agile practices. We will demonstrate when and how to use it.

TF ‘fitting’ – TF is scalable and highly adaptable to the needs of different organisations. Thus you are able to select and apply the right components for each implementation.

Fast changing requirements – TestFrame’s simple Test Cases are easy to maintain and modify in their nature.

Make next step possible – TestFrame’s simple Test Cases enable rapid scale-up to Test Automation and TestFrame Integrator once requirements solidify.

Refactoring – TestFrame Automation framework supports Regression testing as well as the structured scale down to manual testing to accommodate changing requirements. Once requirements solidify again, it is easy to switch back to automated testing.

Rapid Quality Feedback – TestFrame provides continuous Product Quality Monitoring, even throughout a different scaled mix of TestFrame implementations.

Travel light – Light versions of TestFrame allow for a jumpstart with a minimum of documentation and overhead (within the regular TestFrame structure), yet supporting an Agile approach with the ability to report accurately on product quality development.

TestFrame Action Words – provide a smooth transition to and from Automated Testing.

TestFrame Automation – supports continuous testing to keep the system stable as it is built and to support multiple Agile iterations and re-factoring. Short feedback loops –supported by TestFrame test run system and PQM.

Test driven development – guarantees acceptance of the features built.

Joint Test ware Design (JTD) – guarantees the right test conditions are executed by prioritizing on business value and business risk.

Frequent testing gives the customer and the developer’s confidence that the product is evolving in the right direction in both functionality and quality.

Both SCRUM and XP (along with almost all Agile development techniques), mandate quick and frequent feedback and there is no more important feedback than early indications of how well the system works. Acceptance tests allow the customer to assess that the system works and how well the system works. This is why Agile methods advocate ongoing acceptance testing by the customer as part of the iterative cycle. Once there is agreement between Business, Development and Test that the developed software is approaching its final state then a final acceptance test should be performed. These test activities of iteration acceptance and the final and formal acceptance test level activities do complement each other. The test related products such as test cases could well be reused in order to execute the final User Acceptance Test (UAT). Iteration acceptance testing is preparation for the final UAT.

Now let us look into Agile TestFrame ways to reach the goal of final acceptance quickly, effectively and accurate.

TestFrame practices supporting Agile

Final Goal is Acceptance

Agile practices supporting TestFrame


06 A good practice is to begin creating Test Cases as soon as Use Cases (or the SCRUM equivalent of User Stories) are available, and well before any code is written. Using Use Cases or User Stories to generate Test Cases can help significantly to launch the test process quickly and early in the development lifecycle. Use Cases identify and communicate the conditions (however simple) that will be implemented and that are necessary to verify the successful and acceptable implementation of the systems requirements. This gives you a lot of credit, because you do have something to show quickly.

For agility purposes Use Cases and Test Cases should be closely linked. By doing so, the software development and the test development remain synchronized if changes to the requirements specifications occur.

Next we look at TestFrame Action words and how they fit in. A Test Case does include several steps, called ‘actions’ in TestFrame. These steps are documented in a structured and reusable way by previously defined Action Words. An Action Word indicates which action must be carried out for completing the particular step. Along with the Action Arguments test data can be passed like parameters to carry out the Action. This is a very powerful mechanism. It is good practice to name test case actions as an order, for example: ‘Enter client data’, ‘Check client data’. This prepares for the next step: identifying and introducing action words. Action words can be executed manually but are also an important step towards Test Automation. TestFrame uses Action Words to ensure smooth transition to Test Automation.

There are, however, limitations to using Use Cases for Test Case development. Use Cases are used to model functional requirements. They are not used to model non-functional requirements such as capacity and performance. In some cases however non-functional requirements such as response times could be included in the Test Case.

A simple and effective way to start with Test Case design is to derive Test Cases directly from Use Cases. The Test Cases should cover the normal course of the Use Case, alternative courses, and the exceptions. In almost every situation you will discover that it is sufficient to start describing the individual steps and on appropriate steps recording indication of Test Data requirements and adding some expected results.

A good practice is to limit Test Cases to the essentials and even leave out expected results if the expected results are overly obvious. Most users with some support will understand this format and easily grasp what is expected from them. Additional information can be added as required as the project progresses.

In situations where you need to do a Systems Integration Test (verification) it is practical to add, at appropriate steps, a reference to the document and/or paragraph where the agreed functional detail can be found. This can speed up both the verification process and the subsequent addition of test conditions to the Test Case.

From Use Case to Test Case

Action Words

Non-Functional

Start with simple Test Cases



Collaborative workshops can provide a valuable channel for speeding things up. In many circumstances collaborative workshops prove to be a lot more effective and efficient than individual meetings with people. This applies to the initial build and subsequent refinement of your Test Cases too. The Workshop approach can be used successfully in combination with Joint Application Design (JAD), Joint Test Ware Design (JTD) and many other Agile techniques.

Collaborative Test Case refinement workshops have proven to be remarkably successful in reliably producing the desired results – usable Test Cases to begin testing with. Collaborative Test Case refinement workshops will: • Build trust among the workshop participants; business, development and test • Commit users to the Test Case creation process • Create a sense of ownership for deliverables, and ultimately the system, amongst users • Discover both forgotten and non-essential requirements • Shorten the Test Ware creation phase substantially • Reinforce effective communication patterns • Introduce an element of fun into a very serious process

Another proven way to refine your Test Cases is to use defects found. Look regularly at the defects that have been detected. Where did they come from? How could Test Ware be ammended to reveal similar problems in the future more quickly? Using this insight to tighten up your Test Cases will make life easier. Often there is a part of the system where flaws seem to cluster. Use this as an indication to do some thorough checking on that area.

Tracking product quality development is one of many features of TestFrame that can be used to support Agile Testing. In most cases your Stakeholders are not interested in the exact amount of defects found in the software. They are interested in the continuous build-up of quality in the system they have sponsored. TestFrame can provide this even in the most agile environments by tracking the quality per Use Case by simply (hyper)linking simple Test Cases (as described above) into the PQM sheet at the row named TestFrame ‘Test Condition’.

Advantages of a lightweight version of the TestFrame methodology include: • It is adaptable to change. • It is people and expert user oriented rather than process oriented. • It can be used to jumpstart, scale-up and formalise as the testing progresses and the

circumstances demand.

If you work in a SCRUM environment, it is a good practice to align Test Runs with SCRUM Sprints. In doing this, you use the projects rhythm and naturally align to development cycles and timing.

Build and refine Test Cases

using Workshops

Refine Test Cases with defects found

Tracking Product Quality Development


08

5 Agile Test Ware One of the most popular questions you come across about Agile Testing is: how much test ware and documentation do I need? Every artefact that you create, and then decide to keep, will need to be maintained over time, being a real burden if it is over-done or redundant for the purpose. You need to consider the likelihood and impact of change on your test documentation and anticipate accordingly. For example, the details of Use Cases and accordingly the details of Test Cases are the most likely to change in the early iterations of the project. The burden of maintenance therefore will be high in the beginning and lessen towards the final delivery of the project. The decision to use simple Test Cases in the beginning will make you more agile as you are travelling lighter. The more complex or detailed your test cases are, the more likely it is that any given change will be harder to accomplish. You may sometimes even have to down grade to a more simple form of test ware. This practice is particularly important for gaining agility of Test Ware.

In most Agile projects you should consider creating at least a minimum set of Test Ware because of:

Stakeholders require it - One increasingly important reason to document is the business need for compliance. You need to work closely with your stakeholders to determine what they actually need and more importantly, why do they need it? There may be an alternative that will use less effort, or the documentation required for several stakeholders can be combined. You should discuss the form in which specific documented proof is required.

Regulatory Compliancy - this may be driven by Sarbanes Oxley (SOX), BASEL-II, SAS70 or any other external compliancy obligation, as well as compliancy with internal agreed standards. In these situations you need evidence that a defined process has been followed correctly. You still want to create adequate and just good enough – and not more than this - documentation to get the job done; this time including evidence for proof of compliance. A sound piece of advice is to read the appropriate guidelines yourself, because they rarely require what bureaucrats think they require. Be smart about compliance, often there is an alternative that will cost much less in effort. Maybe the compliance officer will accept something like a photograph of a sketch on a whiteboard as proof. Sometimes a digital photo proves more to an auditor than an unsigned report using a lengthy and formal template issued by means of an unsigned e-mail to the project manager.

Interfaces - It is important to define how systems interact with one another, because the interfaces between them are hand-over points of responsibility. Interfaces are a contract that both system owners should mutually agree to, document, and change over time if required.

Support of communication - It is not always possible to locate the test team, the development team and the project stakeholders at the same place. Agile projects in particular need to find ways to communicate Test Ware. However shared documentation is often part of the solution, but it easily leads to misunderstandings, although, it is a good supporting mechanism. Always combine written documentation with occasional face to face discussions, teleconferencing, email and collaborative tools.

Knowledge Management - Another important reason to document is to record knowledge (organizational memory) so passing on lessons learned and best practice to other projects and personnel.

Reasons to document

TOC for a document



Make the next step possible - Future business developments may require the testing of the next major release of your system or to support operations and adaptive maintenance of the version under development. The production of appropriate Test Ware at the start will make both of these tasks easier. It could also be to facilitate the set up for automated regression testing in the next iteration. To enable this, you will want to create just good enough (adequate) quality test ware documentation and supporting materials so that the next step can be effective. You should also consider whether the members of your existing team will be involved in the next effort, the nature of the next phase of the system’s evolution, the nature of that phase and its importance to the organization.

Auditing purposes - One more reason to document is for audit purposes, e.g. proof of existence and that you have complied with mandatory standards. If you cannot prove it, it does not exist. It is of high importance to keep in mind that you can view compliance in two distinctive different ways as H. Gelinck points out in “The Science of Compliance” [Gelinck 2007]. The first way is to view compliance as a burden you have to take on. The other way around is to view compliance rules as a way forward to prove and improve operational excellence. To cash in on these, you should talk with the compliance officer and work with him on creative and agile ways to create proof of compliance. Test Ware to consider 2 Test management

Test Plan – A test plan is required to communicate the test approach and get approval for it from the team and the sponsor. Agile projects often can do with one or two pages. Test Environment specification – At a minimum record the differences between the Agile development environment and the likely life operating environment. In Agile projects this would fit on one single page.

Test specification

On Agile projects this documents are mostly combined with development documents, such as the SCRUM Product Backlog.

Test Design - identifies features to be tested, the test cases to test them and the pass/fail criteria. Test Case - documents the test cases in detail with the actual values used for input and the anticipated outputs. This is considered a must whatever the methodology. Test Scenario - describes the business operational actions. A useful tool to have for Business Acceptance Testing. Test Procedure specification - lists all steps required to operate the system and exercise the specified test cases. If you need it, make it.

Test reporting Test item transmittal report - lists the test items being put into the testing environment. This report only has its value when you do not have a source control system (SCS). Otherwise rely on configuration management or a SCS. Test log - used by the test team to keep track of what occurred during test execution. This may be temporary and does not require maintenance after the test job is completed. Agile projects may use a test log on a dedicated whiteboard. Test incident report – report of findings during the test execution. The main purpose of this report is to communicate issues found and enabling the programmer to resolve the issue quickly. Anything that supports the findings should be included. In most cases screenshots prove to be of great value. Test summary report - summarizes the testing activities, matches outcomes to exit and acceptance criteria and provides input to the management decision to proceed to go live. This report proves particularly valuable if you can extract metrics from it.

2 See also: IEEE Std 829 Standard for Software Test Documentation [IEEE 829]

Compliancy Audits

What to document


10

6 RRBT, TestFrame and Agile lifecycles Within the Agile Community it is common to see teams mix best practices of different agile methodologies, for example SCRUM and XP. As for the tester, the most important things to be aware of in any agile project environment are its iterative lifecycle and the need for frequent communication and feedback. This requires some adjustments on the part of testers: • Do not expect to base your testing on a requirements document signed off at the

beginning of the project. Instead, collaborate with other team members to find out what should be tested. You can decide on the how

• Plan testing and set goals per Agile iteration only, but within the context of the overall project budget and timeframe

• Get used to testing constantly within each iteration - rather than at the end of a development activity

• Decide what and how to test when a product is still unfinished and before development start

• Wait to automate acceptance testing till functionality is almost stable. Component testing should be automated before code is written.

• Collaborate with other team members to find out what to test, rather than testing from requirements documents only

• Participate actively in daily short status meetings and do not restrict your input to test related issues.

This applies to all of RRBT/TestFrame phases. RRBT/TestFrame Phases 1. Preparation Test Analysis Preparation TestManagement file Standards and guidelines Definition of Test clusters and Prioritisation

3. Test execution Test Case execution Regression testing Test Runs and PCM Reporting quality development Issue management

2. Test design and Test Analysis Test Techniques Test cluster analysis

4. Transfer to maintenance Regression test suite

The role of the tester in an Agile project is to add value by thinking about the system from the viewpoint of those who are going to use it but with a grasp of details, possibilities, and constraints of those building it.

A Tester: • Brings a unique combination of business and technical viewpoints and specialized skills • Supports risk management through direction and prioritisation of testing and provides

checks and priority schemes using RRBT • Is more efficient at finding defects, because they focus on it • Finds different types of defects than developers do • Provides a critical view of the system, helping the team avoid complacency and remain

focussed on delivering business value • Facilitates communication both inside and outside the team

The tester role and testing functions must be integrated with the requirements and development teams. An isolated testing function in which testers are involved only at the end of the development phase will not work.

The role of the Tester



Both RRBT TestManagement and TestFrame can be successfully integrated with Agile Software Development Methodologies, Scrum and eXtreme Programming (XP) being the most popular.

Scrum is an Agile software project management methodology aimed at maximizing the business value of the product under development at every stage of that development. XP is a software engineering methodology focusing on developing high-quality code and software systems. XP it self doesn't have management practices.

Scrum and XP overlap at the planning game (XP) and Sprit planning (Scrum). Both encourage similar values and provide complementary practices and rules and support increased communication between project management and developers. Combined, they can provide a structure to boost both the productivity of a team and the quality of its outputs.

RRBT TestManagement complements Scrum by delivering an appropriate, minimal Test Strategy establishing development and test priorities based on both Business Risks and Business Value. Progress reporting is based on risks mitigated and requirements realized. RRBT also provides Test Goals for each Scrum Cycle. TestFrame complements XP by adding Test Case development practices, specialized Test Analysis Techniques, reporting on product quality development and Test Automation with TestFrame Integrator. TestFrame Action Word Testing supports XP with full automation capabilities of both the functional and technical tests.

Scrum project management is distinguished by rapid response to changing requirements, frequent feedback to the customer and the integration of the customer into the project.

Scrum radically differs from traditional management because the planned product (backlog, risk) can be changed at the end of any sprint (iteration), in response to the environment (competition, changing requirements, new insights, new technology, etc.). This ensures that the latest delivered product is always a usable and valuable product to the business.

A Scrum guided software project is controlled by establishing, maintaining, and monitoring key control parameters. These controls are critical to contain areas of uncertainty, unpredictable behaviour, and ultimately avoid the collapse of the team into chaos. Use of these controls is the backbone of the Scrum development process.

Controls used in Scrum: • Risks - the risk associated with a problem, issue, or backlog item • Backlog – a list of all requirements and features that should be available in the completed

product • Issues - Concerns that must be resolved prior to the release of a feature or a backlog item. • Changes - the activities that are performed to resolve a problem or issue • Components - self-contained reusable pieces of software • Packet or Feature Group realisation - a group of software components that will

implement a feature or backlog item. The main Scrum controls are Risk and Backlog (Requirements). RRBT TestManagement complements and strengthens these controls with its Risk & Requirement based approach. Also RRBT Test Monitoring and Test Progress and Quality Reports can be a substantial support for Scrum projects.

Another important characteristic of Scrum is the constant testing and appropriate documentation of a product-as it is built. This enables the team to demonstrate the tested product increment to the customer at the end of each Sprint (Iteration) and getting valuable feedback. This practice also builds confidence and trust across the team and between team and sponsors.

Combining the best

Scrum Project management controls

supported by RRBT


12 SCRUM Initial planning phase is supported by

• TestFrame Test Clusters enabling the testing of Scrum Feature Groups in line with the Test Strategy

• RRBT risk analysis and the identification of relationships between features, risk and quality

SCRUM Planning game is supported by • Alignment of RRBT/TestFrame test clusters to the most valuable Backlog items • RRBT continuous reassessment of risks • RRBT/TestFrame reporting on risks mitigated , test accomplished and ultimately

product fit for going Live. How does TestFrame phasing fit in?

SCRUM RRBT/TestFrame

1. Product Backlog Risk Analysis, Test preparation & Cluster analysis (Release planning)

2. Sprint backlog Cluster Analysis (individual User stories) and choice of appropriate test techniques for cluster

3. Backlog tasks Initial Test Case development

4. SCRUM Cycle Test case execution, Report on Product Quality, Test Case refinement, Test Runs and Regression testing

5. Product increment Test evaluation, Transfer to maintenance

When doing Acceptance tests on traditional software projects it is common to find technical faults at unit or integration level because of code that has never been tested at the unit or integration level. This is a very expensive and inefficient way to find faults that were probably introduced at quite an early stage of the project and can easily consume the time allocated for system and acceptance testing. XP avoids this scenario by one hundred percent component test automation and continuous integration practices.

RRBT and TestFrame inside SCRUM

TestFrame inside XP



Therefore XP discerns two distinctive levels of testing:

Customer level: With the aid of the test members of the team, users develop a minimum set of acceptance tests that assess the overall behaviour of the system. This activity corresponds to the ISTQB test level “Acceptance test’ (UAT and BAT). For testing at this level, using light versions of TestFrame through to TestFrame Automation using TestFrame Integrator prove of great value. Whilst developing adoptions are accommodated, after acceptance by the customer the TestFrame light version of a Test Case can be quickly automated.

Programmer level: Component tests should be automated before component code is written. This activity corresponds to ISTQB test level ‘component test and component integration test’. Often component tests are created by using pair programming. One programmer concentrates on developing the component test (for example in JUnit) while the other developer codes the software component.

The Testers Role in XP is to define, automate (if appropriate), and run acceptance tests. When someone is focused on that role it allows focus in other areas as well. The XP Tester also supports the whole team with his test expertise (for example specialized Test techniques).

How does TestFrame phasing fit in?

XP RRBT/TestFrame

1. Release Planning Risk Analysis, Test preparation & Cluster analysis (Release planning). The Tester negotiates quality with the customer and helps to Estimate acceptance-test time.

2. User Stories Planning Cluster Analysis (individual User stories) and choice of appropriate test techniques for cluster testing.

3. User Stories Creation Initial Test Case development, create effective tests before coding begins. Identify and make explicit hidden assumptions in the User Stories. The Tester helps to clarify the proposed User Stories and adds Variants and exceptions to it

4. Acceptance Tests Test case execution, Report on Quality, Test Case refinement, Test Runs and Regression testing. The XP Tester makes sure acceptance tests meet the quality negotiated with and specified by the customer. This includes: functional, load, stress, performance, compatibility, security, and anything not covered by unit and integration tests. Once an acceptance test passes, it should never fail on re-tests.

5. Small Releases Test evaluation and Transfer to maintenance

Customer and Acceptance

Testers Role


14

7 Conclusion It is clear to see that both RRBT TestManagement and TestFrame can be integrated to and add enhance projects run using Agile methodologies, techniques and practices. RRBT TestManagement and TestFrame are robust and scalable, thus addressing the needs of both plan- driven and agile methods, as well as being adaptable to a wide range of environments and situations.

RRBT adds strategy and reporting on risks. TestFrame supports Agile with Acceptance testing and Product Quality Monitoring.

The combination of Scrum, RRBT TestManagement, XP and TestFrame provides the ‘just enough’ structure, process and control that is a key pre-requisite for Agile success.



A Glossary of terms Action word - a combination of action on the test object, defined by the test analyst, which describes how test rules must be carried out (definition from TF course). Agility – Agility is the ability to both create and respond to change in order to profit in a turbulent business environment. Agility is the ability to balance flexibility and stability. (Highsmith 2002)

Continuous integration - additions and modifications to the code are integrated into the build system on at least a daily basis. Automated unit regression tests are used to ensure additions or modifications do not introduce new defects to previously accepted code.

Refactoring – Removing duplication and needless complexity from the code during each coding session without changing functionality, even when this requires modifying components that are already complete and accepted by the user. Automated tests should be used to verify that change does not break functionality.

SCRUM sprint - A time period (usually 2 to 4 weeks) in which development occurs on a set of stories that the Agile team has committed to.

Test Case - a set of input values, execution preconditions, expected results and execution post conditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement. [ISTQB, after IEEE 610]

Test Driven Design (TDD) - technique that involves repeatedly first writing a test case and then implementing only the code necessary to pass the test.


16

B Recommended reading [Augano 2004] Managing Agile Projects, Kevin Augano, PMP, MAPM, The Project

Management Essentials Library, ISBN 1-895186-11-0

[Beck 2000] Beck, Kent, Extreme Programming Explained: Embrace Change ISBN 0201616416

[Beck 2003] Beck, Kent, Test Driven Development: By Example, ISBN 0321-14653-0

[Buwalda 2001] Integrated Test Design and Automation, Hans Buwalda, Dennis Janssen, Iris Pinkster, ISBN 0-201-73725-6

[Cohn 2005] Agile Estimating and Planning, Mike Cohn, ISBN 0131479415

[Cohn 2004] User Stories Applied: For Agile Software Development, Mike Cohn , ISBN 0321205685

[Crispin 2002] Testing Extreme Programming, Lisa Crispin, ISBN : 0-321-11355-1

[Fowler ] Fowler, Martin, Refactoring: Improving the Design of Existing Code, ISBN 0-201-48567-2

[Gelinck 2007] Gelinck, Henriette, The Science of Compliance, ISBN 90-78907860302-x

[Gottesdiener 2002] Requirements by Collaboration, Workshops for Defining Needs, Ellen Gottesdiener, ISBN 0-201-78606-0

[IEEE 829] IEEE Std 829, IEEE Standard for Software Test Documentation.

[Koskela 2008] Test Driven, Practical TDD and Acceptance TDD, L. Koskela, ISBN 1-932394-85-0

[Kroll 2006] Agility and Discipline Made Easy: Practices from OpenUP and RUP Kroll, Bruce MacIsaac, ISBN 978-0-321-32130-5

[Pinkster 2006] Successful Testmanagement an integral approach, (2nd print 2006) Iris Pinkster, Bob van der Burgt, Dennis Janssen, Erik van Veenendaal ISBN: 3-540-22822-5

[Schwaber 2002], Agile Software Development with SCRUM Ken Schwaber, Mike Beedle , ISBN 0130676349

[Wiegers 2003] Software Requirements, Second Edition, Karl E. Wiegers, ISBN:0735618798

[Wiegers 2006] More about Software Requirements—Thorny Issues and Practical Advice, Karl E. Wiegers, ISBN 0-73562-267-1

[Yourdon 2004] Death March, second edition, Edward Yourdon, ISBN 0-13-143635-x


LogicaCMG Nederland BV Kralingseweg 241-249 3062 CE Rotterdam T: +31 (0)10 253 7000 F: +31 (0)10 253 7001 www.logicacmg.com

whitepaper agile rrbt+tf

Documents