inside iso 29119 exposed webinar with jon d. hagar - xbosoft
Post on 03-Aug-2015
59 Views
Preview:
TRANSCRIPT
XBOSoft, Inc. All Rights Reserved. 1
XBOSOFT Inside ISO 29119 Exposed with Jon Hagar, Senior Consultant #ISO29119 #InsideISO
XBOSoft Dedicated to software quality improvement
Founded in 2006
We speed products to market with our expert:
• Software QA consulting
• Software testing
Global team with offices in San Francisco, Beijing and Amsterdam
“Thorough, accurate and fast”
XBOSoft, Inc. All Rights Reserved. 2
House Rules Participants other than the speakers are muted
Questions via the GoToWebinar control on the right side of your screen or through Twitter @XBOSoft
Questions may be asked throughout the webinar - we’ll try to answer them at the end
You will receive info on recording after the webinar
#ISO29119
#InsideISO
XBOSoft, Inc. All Rights Reserved. 3
Meet Our Speakers
Phillip Lew CEO and Founder , XBOSoft • Relevant specialties and
passions o Software quality process,
evaluation, measurement and improvement
o Software quality in use / UX design
o Mobile User Experience and usability
o Cycling and travel
XBOSoft, Inc. All Rights Reserved. 4
Karla Smith Marketing Manager, XBOSoft • Emails you about our
industry hot topic webinars
• Invites you to download our latest trends in software testing whitepapers
XBOSoft Team
Jon Hagar
XBOSoft, Inc. All Rights Reserved. 5
• More than thirty years’ experience in software
and testing • Systems-software tester consultant and trainer
• Supports software product integrity, testing,
verification, and validation with a specialization in embedded, IoT, and mobile software systems
• Projects supported include control system (avionics and automotive), mobile-smart devices, and ground systems (IT and PC).
• Publishes and writes regularly including his book: Software Test Attacks to Break Mobile and Embedded Devices, CRC press.
• Lead editor/author on numerous international standards including ISO 291119 software testing.
Copyright 2013 Jon Hagar Grand Software Testing GST
Jon D. Hagar, Senior Consultant embedded@ecentral.com
Author: Software Test Attacks to Break Mobile and Embedded Devices
Copyright 2015, Jon D. Hagar Grand Software Testing, LLC – “Software Test Attacks to Break Mobile and Embedded Devices”
6
Copyright 2013 Jon Hagar Grand Software Testing GST
Current Situation
Controversy (https://www.youtube.com/watch?v=RCl8lGxY0WU)
Many people do NOT know much about it
Starting to go into use in places in the world
Copyright 2013 Jon Hagar Grand Software Testing GST
Scope
Purpose of standards Motivation for ISO 29119 standards Overview of ISO29119 Applicability Thoughts
Q&A
Copyright 2013 Jon Hagar Grand Software Testing GST
Some Preliminaries Under disclosure, I am the IEEE Project Editor for
ISO/IEEE/IEC 29119 Software Testing Standard
For many organizations: Standards are a fact of life for many industry professionals
not just testers
Standards form part of the business for many companies , customers, and countries
We end up in a mixed relationship with standards
Standards should be viewed as a baseline
Copyright 2013 Jon Hagar Grand Software Testing GST
What are standards?
Guideline documents—not compulsory unless mandated
Agreements because they often reflect a certain level of consensus within a community
“Guideline documentation that reflects agreements on products, practices, or operations by nationally or internationally recognized industrial, professional, trade associations or governmental bodies” - ISO
Copyright 2013 Jon Hagar Grand Software Testing GST
How can a standard “help?”
Standards are a basis for: Communication – common terminology (trading language) Professionalism and criticism Conformance/compliance schemes Benchmark of ‘industry practice’ (not best, may not be good
for all, etc.) Contracts and regulations Interoperability and consistency. . .
Standards describe a current ‘body of knowledge’ that provides the basis for a professional discipline
Copyright 2013 Jon Hagar Grand Software Testing GST
Poll
Would you consider using a standard if it solved problems your team might have and could be tailored to better fit your needs? Yes No
Copyright 2013 Jon Hagar Grand Software Testing GST
Motivation for ISO29119 Conflicts in definitions, processes and procedures
“One ring to rule them all” — old testing standards to be replaced by ISO 29119 e.g., IEEE 829, IEEE 1008, BS7925-1/-2, IEEE 1028
Lacking in current standards
Organizational areas e.g., Test Policy and Organizational Test Strategy
Project Test Management BS7925 only covers unit testing General processes Common functional techniques missing Coverage of non-functional testing
Part of the larger ISO/IEEE family of systems and software standard, e.g. ISO 12207, 15288, IEEE1012
Copyright 2013 Jon Hagar Grand Software Testing GST
Standardization Bodies
Copyright 2013 Jon Hagar Grand Software Testing GST
ISO/IEC/IEEE 29119 –Structure and History
BS7925-1
BS7925-2 IEEE 829
Concepts & Vocabulary Part 1
Process Assessment
Testing Techniques
Part 4
Documentation
Part 3 Part 2
Processes
Keyword-Driven Testing
Part 5 ISO/IEC 33063
ISO 12207 ISO 15288 Directives
IEEE 1008
Copyright 2013 Jon Hagar Grand Software Testing GST
Part 1: Concepts & Vocabulary
SOFTWARE TESTING CONCEPTS
Scope, Conformance, Normative References
TESTING IN DIFFERENT LIFE CYCLE MODELS
ROLES AND RESPONSIBILITIES IN TESTING
ANNEXES – Metrics, Examples, Bibliography
DEFINITIONS
Test
: A
ppro
ach,
Bas
is,
Met
hods
-
Ris
k Bas
ed T
estin
g
Copyright 2013 Jon Hagar Grand Software Testing GST
Part 2: Testing Processes
TEST MANAGEMENT PROCESSES
ORGANIZATIONAL TEST PROCESS
DYNAMIC TEST PROCESSES
Copyright 2013 Jon Hagar Grand Software Testing GST
TEST MANAGEMENT PROCESSES
ORGANIZATIONAL TEST PROCESS
DYNAMIC TEST PROCESSES
Instantiating Testing Processes
Ref: S. Reid
Copyright 2013 Jon Hagar Grand Software Testing GST
Organizational Test Specification
Develop Organizational
Test Specification
(OT1)
Monitor and Control Use of Organizational
Test Specification
(OT2)
Updated Organizational Test Specification
Update Organizational
Test Specification
(OT3)
Controlled Organizational Test Specification
Organizational Test Processes
Copyright 2013 Jon Hagar Grand Software Testing GST
ORGANIZATIONAL TEST PROCESS
TEST MANAGEMENT PROCESSES
TEST PLANNING
TEST MONITORING &
CONTROL
TEST COMPLETION
ORGANIZATIONAL TEST
DOCUMENTATION
FEEDBACK ON ORGANIZATIONAL TEST DOCUMENTATION
TEST PLAN UPDATES
TEST PLAN
TEST COMPLETION REPORT
DYNAMIC TEST PROCESSES
TEST MANAGEMENT
PROCESSES
TEST PLAN, TEST COMPLETION REPORT, TEST MEASURES
TEST MEASURES
TEST PLAN, CONTROL DIRECTIVES
TEST PLAN, CONTROL DIRECTIVES
Test Management Processes
Copyright 2013 Jon Hagar Grand Software Testing GST
Organize Test Plan
Development Identify &
Estimate Risks
Design Test Strategy
Determine Staffing and Scheduling
Document Test Plan
Schedule, Staffing Profile
Test Strategy
Analyzed Risks
Scope
Identify Risk Treatment Approaches
Gain Consensus on
Test Plan
Approved Test Plan
Draft Test Plan
Test Plan Publish
Test Plan
Understand Context
Treatment Approaches
Test Planning Processes
Copyright 2013 Jon Hagar Grand Software Testing GST
Part 2 Standard Primarily has a Risk-based Process Flavor
Risk based testing is a primarily “good” practice
Other bases/approaches (Agile, exploratory, model-based, math-based, etc.) are included
All parts may need “tailoring” (few efforts would be “full” conformance)
Use of the standard implies skills, thinking, dependence on context (“it depends”), etc.
Copyright 2013 Jon Hagar Grand Software Testing GST
Test Monitoring & Control Process
Monitor
Report
measures
Control Set-Up
[testing complete]
[testing incomplete]
control directives
test progress
info
test progress info
test plan
test measures
test status report
test control info
...TEST PROCESSES... Dynamic/Management
Copyright 2013 Jon Hagar Grand Software Testing GST
DYNAMIC TEST PROCESSES
TEST SPECIFICATION
[NO ISSUES NOTICED]
TEST ENVIRONMENT READINESS REPORT
TEST ENVIRONMENT
REQUIREMENTS
INCIDENTREPORT
TEST DESIGN & IMPLEMENTATION
TEST ENVIRONMENT
SET-UP
TEST EXECUTION
TEST INCIDENT
REPORTING
(LEVEL) TEST MANAGEMENT PROCESS
[ISSUE NOTICED or RETEST RESULT]
TEST RESULTS
CONTROL DIRECTIVES TEST
MEASURES
(LEVEL) TEST PLAN
Dynamic Test Processes
Copyright 2013 Jon Hagar Grand Software Testing GST
Part 3 – Test Documentation
TEST DOCUMENTATION
ANNEXES - EXAMPLES
Scope, Conformance, Normative References
Sel
ect
a su
bset
of
docs
Copyright 2013 Jon Hagar Grand Software Testing GST
Part 3: Test Documentation
Organizational test documentation Test policy Test strategy
Project test documentation Project test plan Test project completion report
Test Level documentation Test plan Test specification Test results Anomaly reports Level test status report Test environment report Test level completion report
Appendices Examples of documents at each level of testing
Not all need to be produced when tailoring is used- Most will NOT produce all of these
Copyright 2013 Jon Hagar Grand Software Testing GST
Part 4 – Test Techniques (possible publish in 2015)
TEST COVERAGE MEASUREMENT
Scope, Conformance, Normative References
ANNEXE – TESTING OF QUALITY CHARACTERISTICS
ANNEXE – SELECTION OF TECHNIQUES
ANNEXE – TEST TECHNIQUE EFFECTIVENESS
TEST DESIGN TECHNIQUES
Functional Structural
Copyright 2013 Jon Hagar Grand Software Testing GST
Part 5- Keyword-Driven Testing (possible publish in 2016-17)
Part 5 addresses:
Concept
Applicability
Interfaces
Approach
Part 5 in review and ballot
Copyright 2013 Jon Hagar Grand Software Testing GST
Applicability of ISO/IEEE 29119 It is intended to be a generic standard to support different
environments and domains of testing when selected
Some domains are more standards oriented such as: Safety-related Telecoms International business and large companies
Some domains are more regulated world-wide such as:
Financial – banks, stock markets, etc. Medical
Can support various lifecycle models
Tailored to sequential, iterative and agile (highly tailored) Applicability to new as well as maintenance projects
Copyright 2013 Jon Hagar Grand Software Testing GST
Organization where the Standard Might Be Used
Providing users a common baseline of testing concepts, terms, and ideas Trading Language
Government organizations and regulators needing an industry approved starting baseline
Purchasers and procurers of software products to assess third parties Developers of software, e.g. to support unit level testing. Training course syllabus developers and providers University lecturers wishing to teach from an ISO standards University researchers wishing to conduct research Software testing tool vendors Organizations and regulators using the standard for accreditation International business and organization who use other ISO standards
Copyright 2013 Jon Hagar Grand Software Testing GST
Do Testers Need Standards? – Not all, but some may Standards support common communication within the topic
Common reference points Starting point for usage
Maturity of the standard is an issue but a baseline serves as
sounding board and common reference point for “Scientific” method An international benchmark Thinkers and researchers can prove/disprove benchmark(s)
Part of being in a profession (but only part)
Standards are not for everyone
Copyright 2013 Jon Hagar Grand Software Testing GST
ISO29119 Standard - My Viewpoint Any software standard is based on limited history (rear
looking) and developed based on consensus of many voices Represent only some of viewpoints of industry
Will expand to include more testing ideas
Not perfect
Most using projects/professionals will use “tailoring”
Copyright 2013 Jon Hagar Grand Software Testing GST
My Thoughts on how to tailor Thinking testers with a context-driven/based viewpoint
Keep “good” (worthwhile)
Remove or change what is not worthwhile
Use standards as a professional reference work, but
know when and how to break the “rules” via tailoring
Get stakeholder buy in – Fight to Educate
Copyright 2013 Jon Hagar Grand Software Testing GST
Summary First attempt at a comprehensive software test
standard
Even in the presence of standards, a thinking tester is always needed
Get it read it, tailor it (as needed), and form your own opinions by “testing it” yourself Send me your thoughts and changes
Not universal, best, or comprehensive
Copyright 2013 Jon Hagar Grand Software Testing GST
For more information
jon.d.hagar@gmail.com For more information and details
http://softwaretestingstandard.org/ WG26 website
http://www.jtc1-sc7.org/ Access to official documents released by WG 26
Copyright 2013 Jon Hagar Grand Software Testing GST
Stuart Reid
Testing Solutions Group 117 Houndsditch
London EC3 UK
Tel: 0207 469 1500 Fax: 0207 623 8459
www.testing-solutions.com
Major Reused Reference Point:
Post your questions on Twitter and we'll answer them @XBOSoft Join us to keep updated on all our webinars, reports and whitepapers: facebook.com/xbosoft +xbosoft linkedin.com/company/xbosoft We post regularly on our blog – check us out! http://xbosoft.com/software-quality-blog/ Download our free whitepapers, available here: http://xbosoft.com/knowledge-center/ You will receive an email with information on today’s webinar slides and recording. Any further queries regarding our services or ideas for future webinars, please email us! Services@xbosoft.com
Q+A
www.xbosoft.com
top related