1
ISTQB and Test AutomationHow ISTQB Certification Prepares youfor Automated Testing
Presented by
ANDREW L POLLNER
Vice President, ASTQBChair, ISTQB Expert Level Test Automation Working Group
ASTQB Webinar Series
Copyright © 2014 ASTQB Slide 3
IntroductionISTQB and ASTQBImportance of Automation in TestingAutomation Topics in ISTQB SyllabiBusiness OutcomesExpert Level Syllabi
Copyright © 2014 ASTQB Slide 4
WORLD-WIDE COVERAGE
Countries covered by Member Boards Countries covered by Exam Providers
ASTQB
Copyright © 2014 ASTQB Slide 5
Goals for Automation in Testingo Fast execution of testso Repeatable processo Increased functional coverageo Lots of variations and permutationso Automatic reporting of errorso Low maintenance requirementso Make test team more productiveo Stay on time and on budget
Copyright © 2014 ASTQB Slide 6
Body of Knowledge (BOK)What Documents Cover Automation?o Foundation Levelo Advanced Level Test Analysto Advanced Level Technical Test Analysto Advanced Level Test Managero Expert Levelo Glossary
6
Copyright © 2014 ASTQB Slide 7
Foundation LevelChapter 6 – Tool Support for Testing:o Types of Test Toolso Effective use of Toolso Introducing a Tool into an Organization
Copyright © 2014 ASTQB Slide 8
Advanced Level Test AnalystChapter 7 – Test Tools:o Test Data Preparation Toolso Automated Test Execution Toolso Test Automation Implementationo Improving the Success of the Automation Efforto Keyword-Driven Automationo Causes for Failures of the Automation Effort
Copyright © 2014 ASTQB Slide 9
Advanced Level Technical Test Analyst
Chapter 6 – Test Tools and Automation:o Integration and Information Interchange
Between Tools o Defining the Test Automation Project o Selecting the Automation Approacho Modeling Business Processes for Automation o Specific Test Tools o Tools for Web-Based Testing
Copyright © 2014 ASTQB Slide 10
Advanced Level Test ManagerChapter 6 – Test Tools and Automation:o Tool Selectiono Selection Processo Return on Investment (ROI)o Tool Lifecycleo Tool Metrics
Copyright © 2014 ASTQB Slide 11
Expert Level Test AutomationTwo Modules:o Test Automation Engineeringo Test Automation Management
Copyright © 2014 ASTQB Slide 12
Business Outcomes – TAEo Contribute to the development of a plan to integrate
automated testing within the testing processo Evaluate tools and technology for automation best fit to
each project and organizationo Create an approach and methodology for building a test
automation architectureo Design and develop new or modified test automation
solutions that meet the business needso Create automated test reporting and metrics collectiono Manage and optimize testing assets to facilitate
maintainability and address evolving test systems
Copyright © 2014 ASTQB Slide 13
Business Outcomes – TAMo Develop a test automation policy and strategy appropriate for
the organizationo Create a business case which outlines the costs and benefits
to be expected from introducing and sustaining test automation
o Develop a plan to integrate automated testing within the testing process
o Oversee the evaluation of tools and technology for automation best fit to each project and organization
o Manage the test automation resources to implement and sustain an effective test automation program
o Define and gather metrics to effectively report progress of the test automation program
o Apply findings to improve the test automation process
Test Automation Engineering (TAE)In Scopeo Tasks performed by a test
automation engineer o Designing, developing and
maintaining concrete test automation solutions
o Concepts, methods, tools, and processes
o Automating dynamic functional tests
Out of Scopeo Test Managemento Automation of non-functional
testso Teaching of software
engineering methods and programming
o Teaching of software technologies
o Selection of software testing products and services
Copyright © 2014 ASTQB Slide 15
Topics in TAEo Introduction and Objectives for Test Automationo Preparing for Test Automationo The Generic Test Automation Architectureo Deployment Risks and Contingencieso Test Automation Reporting Metricso Transitioning Manual Testing to an Automated
Environmento Verifying the TASo Continuous Improvement
Copyright © 2014 ASTQB Slide 16
SDLC processes for SUT and TAS
Copyright © 2014 ASTQB Slide 17
K LEVELS: Syllabus & ExamQuestions are classified according to the cognitive level, the K-level (also known as level of knowledge):K1 – Remember (recognize a term or concept)
K2 – Understand (able to explain a concept)
K3 – Apply (select correct application of concept or technique)
K4 – Analyze (can distinguish between facts and inferences for deeper understanding)
K5 – Evaluate (can make judgments based on criteria and standards)
K6 – Create (can put elements together to form a coherent or functional whole)
The number of questions for each topic is related to the length of the topic in the syllabus
Copyright © 2014 ASTQB Slide 18
Learning Objectives (Chap 1-2)1.1 Purpose of Test Automation
ELTA-E-1.1.1 (K2) Explain the objectives, advantages and limitations of test automation
2.1 Test Automation Requirements for the SUT and its Context ELTA-E-2.1.1 (K4) Analyze a system under test to determine the optimal automation solution
Copyright © 2014 ASTQB Slide 19
Learning Objectives (Chap 3-4)3.2 TAA Design
ELTA-E-3.2.1 (K4) Design the appropriate TAA for a given project
3.3 TAS Development ELTA-E-3.3.1 (K6) Construct a purpose-built TAA based on the generic TAA (gTAA) using predefined components
4.3 Technical Risk Assessment and Mitigation Strategies ELTA-E-4.3.1 (K6) Devise mitigation strategies to address technical risks, including framework approach, that could lead to failure of the test automation project
Copyright © 2014 ASTQB Slide 20
Generic Test Automation Architecture
Copyright © 2014 ASTQB Slide 21
Learning Objectives (Chap 5-6)5.3 Logging of the TAS and the SUT
ELTA-E-5.3.1 (K4) Implement logging of test executions and test results (of both TAA and SUT data)
6.1 Criteria for Automation ELTA-E-6.1.1 (K4) Analyze factors relevant to determining the appropriate criteria for suitability of tests for automation
Copyright © 2014 ASTQB Slide 22
Learning Objectives (Chap 7-8)7.1 Verifying Automated Test Environment
Components ELTA-E-7.1.1 (K3) Apply validity checks to the automated test environment and test tool setup
8.2 Planning the Implementation of Test Automation Improvement ELTA-E-8.2.1 (K4) Analyze the test environment components, including tools, in order to understand where consolidation and updates need to be made following a given set of test environment changes
Copyright © 2014 ASTQB Slide 23
Test Automation ManagementTentative Topics:o Test Automation Policyo Test Automation Strategyo Managing Test Automationo Test Automation Resources
Copyright © 2014 ASTQB Slide 24
ISTQB LEVELS AND SYLLABI
Syllabus in Progress
Syllabus ReleasedExam in Progress
Certification Prerequisite
Copyright © 2014 ASTQB Slide 26
ISTQB CERTIFICATES ISSUED
Copyright © 2014 ASTQB Slide 27
Conclusiono Test Automation is more similar to software
development than to testingo ISTQB syllabi provide a wealth of information on Test
Automation topics and a forthcoming certification examo Certification criteria:o 7 Years of practical testing experienceo Foundation Level Certificationo Advanced Level TTA Certification (for TAE)
Copyright © 2014 ASTQB Slide 28
Questions?