10 th t3uc 2011conference, 7-9 june 2011, bled, slovenia invited talk: future testing challenges

T3UC/Bled-Slovenia/ 9th June 2011

10th T3UC 2011Conference,7-9 June 2011, Bled, Slovenia

Invited Talk:Future Testing Challenges

Arunkumar KhannurISQT Process and Consulting Services Pvt Ltd#732, I Floor, 12th Main, III Block, Rajajinagar,

BANGALORE- 560 010 INDIAEmail: [email protected]

www.isqtinternational.comPh: +91-80-23145564/65

1www.isqtinternational.com

http://images.google.co.in/imgres?imgurl=http://api.ning.com/files/jIBxd0lHFDxrNUgoQ0bksXjZDfLHfUmindDsbxtNz1AmyEujX4Xetfo3j1GS5zEs0lxEqZYIUAMPtx*jUiwvqlrHQYruYurw/reva.png%3Fcrop%3D1%253A1%26width%3D171&imgrefurl=http://osum.sun.com/group/revainstituteoftechnologyandmanagementbangalore&usg=__xqzwEgEHDDHe_64G768dp81Ggq4=&h=171&w=171&sz=26&hl=en&start=7&um=1&tbnid=mM4aARmjN_-tkM:&tbnh=100&tbnw=100&prev=/images%3Fq%3Dreva%2Binstitute%2Bof%2Btechnology%26hl%3Den%26sa%3DN%26um%3D1



http://images.google.co.in/imgres?imgurl=http://www.hmsit.ac.in/files/vtu.jpg&imgrefurl=http://www.hmsit.ac.in/link.php%3Faction%3DSave%26linkid%3D9%26name%3DAcademic&usg=__4eey0Xlf3VAZeBx-Jxh8QfTi7YQ=&h=514&w=359&sz=131&hl=en&start=2&um=1&tbnid=ds5z24edu604JM:&tbnh=131&tbnw=91&prev=/images%3Fq%3DVTU%26hl%3Den%26sa%3DG%26um%3D1


Points that We Discuss

Initial remarksSoftware testing challengesWhat is real and what is represented always differAddressing an unreliability of our perceptionsWhere are we heading?Top 3 test challenges, risks, and solutions

www.isqtinternational.com

2


Initial Remarks

Software Testing is a process of measuring quality of a softwareSoftware Testing has many challenges


3


Software Testing Challenges

Quality is relativeTwo types of quality– Primary or Intrinsic

Ex: Functional aspects– Secondary or Extrinsic

Ex: Non-functional aspectsUnrealistic Requirements Specifications


4


Software Testing Challenges

Increasing Technical ComplexityIncreasing Managerial ComplexityTest related Standards and Frameworks are yet to matureImmaturity in Book of Knowledge


5


What is Real and What is Represented always Differ

what is real and what is represented always differ. – Requirements specifications is

a "representational" perceptual model that is built based on inner "ideas", "impressions' or "sense data" of an observer (requirements study team member) and his inferences

Thus, most of the errors in software have origin in requirements


6


How can We have Better Representation of Requirements?

If we find a way to directly establish a link between the observer's inner world and external object, we can have better representation of requirements. However, major hurdle to achieve this is unreliability of our perceptions


7


Addressing an Unreliability of our Perceptions

To reduce the gap between perceptual model of our inner ideas and outer objects , we need to understand that there are two types of qualities, namely, – Primary Quality (Absolute Quality) and – Secondary Quality (Relative Quality).


8


Primary Vs Secondary Quality

What is real and what is represented always differ– If we find a way to directly establish a

link between the observer's inner world and external object, we can have better representation of requirements.

– However, major hurdle to achieve this is unreliability of our perceptions.


9


Secondary Quality

Secondary Quality– Qualities of any object that are extrinsic

(not intrinsic) and vary based on time and conditions and as such which define relative quality

– Our ideas that we develop in our mind on secondary qualities never resemble the the physical object itself


10


Example: Secondary Quality

The colour of the User Interface is not a property of the screen itself but a product of – the interaction of various factors, including

certain physical attributes of the screen such as power supply, resolution,

– the peculiarities of our own sensory system; and

– the environmental conditions prevailing at the time of the observation


11


Primary Quality

Primary Quality– Qualities of any object that are intrinsic and do

not vary based on time, perception and conditions and as such which define absolute quality.

– our ideas that we develop in our mind on primary qualities closely resemble the physical object itself

– Thus, primary qualities of physical objects define absolute quality

Primary qualities also help us – in explaining and also, – developing an experience of the secondary

qualities


12


Example: Primary Quality

Example– User Interface screen has certain true

properties which are intrinsic, such as its size and shape, which do not depend on the conditions under which the screen is observed or on the existence of the viewer


13


Primary Quality and Requirements Specification

While capturing requirements always think on primary qualities of clients wants and needs. – If you are able to identify such primary

qualities, then you can have concrete requirements that are beyond scepticism.

Such requirements which can be represented using primary qualities are implementable and measurable. For example, requirements like accuracy of numbers, length of any text field, number of permissible users, number of transactions that need to be supported by the system etc are primary qualities


14


Secondary Quality and Requirements Specification

Secondary qualities of requirements can not be concrete and as such they are the basis for skepticism. Secondary qualities can not be implemented to perfection and also, can not be measured. For example, requirements like system shall be user friendly, system shall have recoverability feature, user interfaces shall be pleasing etc are secondary qualities.Thus, while arriving at requirements specification if we focus on primary qualities then our requirements will be concrete. Else requirements will be representation full of scepticism.


15


Focus during Requirements Specification

Thus, while arriving at requirements specification if we focus on primary qualities then our requirements will be concrete. Else requirements representation will be full of scepticism


16


IncreasingTechnical Complexity







Increasing in Technical Complexity


18


Where are We Heading?







Point 1: From QC to QA to QE


20


Point 2: Transformation in Approach

At Present– Process Centric

(Exploitation)– Plan and Design

Centric– Focus is on Test

Coverage (Efficiency Centric)

– Test Acceptance Criteria is “Zero Defect” centric

– Focus is on Return on Investment (RoI)

Future– Exploratory, Lean,

Agile, and Risk based

– Strategy and Architecture

– Focus will be on Reducing Risks (Effective Centric)

– Test Acceptance Criteria will be “Manageable Risk” centric

– Focus is on Return on Expectation(RoE)



Characteristics of Exploration and Exploitation

Characteristic Exploration ExploitationTesting Focus Invention of testing Administration of

TestingOverriding Goal

Dynamically moving from current knowledge to the next

Systematically honing and refining within the current knowledge stage

Driving Forces

Intuition, Feeling, Hypotheses about the future, Originality

Analysis, Reasoning, Data from the Past, Mastery

Future Orientation

Long Term Short Term

Progress Uneven, Scattered, Characterized by False Starts and Significant leaps forward

Accomplished by measured, careful incremental steps

Risk and Reward

High Risk, Uncertain but Potentially High Reward

Minimal Risk, Predictable but Smaller Rewards

Challenge Failure to consolidate and exploit returns

Exhaustion and obsolescence



Effort towards Value Creation


23

Analytical Thinking

Intuitive Thinking

Declaring Truth and

Certainties

Creativity and

Innovation

Design Thinking


Point 3: Effective STLC

At Present: STLC– Plan and Design

CentricScopingPlanningDesigningExecutionDefect

ManagementClosure

Future: STLC– Strategy and

Architecture Centric

Test AnalysisQuality ProfilingModelingStrategyTest Approach

SelectionRisk

ManagementClosure



Arriving at Test Strategy: Preparatory Work

Test Architect/ISQT/V1.0

Requirements Specification

Define TestScope

Identify Test Conditions

Identify Test Oracles

Identify Test Basis

Understand Context

QualityProfiling

TEST STRATEGY

RequirementsReview

Feasibility Analysis

Test Requirements Elicitation

Test Analysis

Defining Test Procedures


Point 4: Dominance of Model based Approaches

Modeling of Requirements– Visual Modeling– Quality ProfilingModel based Testing Techniques– Use-case based– State Transition Diagram based– Decision Table based– Etc


26


Representing Expected Quality Profile from Requirements

Test Architect/ISQT/V1.0

Requirements

Business

Non-Functional

Functional

Technical

Functionality

Reliability

Efficiency

Usability

Portability

Maintainability

AccuracyComplianceInteroperabilitySecuritySuitability

AnalysabilityChangeabilityStabilityTestability

AdaptabilityInstallabilityReplaceability

Fault toleranceMaturityRecoverability

LearnabilityOperabilityUnderstandability

Resource behaviorTime behavior


Point 5: Transformation in Test Automation

Model based Test Automation Tools bacome more and more popular

www.isqtinternational.com 28


Point 6: Non-functionality Testing

Non-functional testing draws more and more attention– Quite difficult to automate– Major reason for Customer dis-

satisfaction– Solution is in Quality Engineering


29


Point 7: Evolution of Standards and Frameworks

Standards and frameworks receive major attention– Book of Knowledge– Test Maturity Models– Product Quality Standards– Governance-Risk-Compliance related

standards– etc


30


Top 3 Test Challenges, Risks, and Solutions








Challenge 1:– “Delivery Deadline” Syndrome will

continueRisks– Lesser test effectiveness– More stress on testing teamSolution– Model Based Testing will be effective

Optimum test casesIncreased test effectivenessStrategy to select few test conditions and lesser rounds of testing


32



Challenge 2:– Using CMMI like process for testingRisks– “Process Overhead”– “Too many metrics with lesser

relevance”Solution– Switch over to TMM based processes– Focus on risks than on metrics

Stability


33



Challenge 3:– Non-functional testing will be the major

area of focusRisks– Not enough skilled professionals– AttritionSolution– Continuous training to prepare a pool of

non-functional professional– Back-up for every role


34


Questions ?







THANK YOU






10 th t3uc 2011conference, 7-9 june 2011, bled, slovenia invited talk: future testing challenges

Documents