detection theory applied to finding and fixing defects

Post on 21-Jan-2018

154 Views

Category:

Software

1 Downloads

Preview:

Click to see full reader

TRANSCRIPT

BT6 ConcurrentSession11/12/1511:30am

“Detection Theory Applied to Finding and Fixing Defects”

Presented by:

Ru Cindrea

Altom Consulting

Broughttoyouby:

340CorporateWay,Suite300,OrangePark,FL32073888-268-8770·904-278-0524·info@techwell.com·www.techwell.com

Ru Cindrea Altom Consulting

Ru Cindrea (@ru_altom) is a senior test consultant and managing partner at Altom Consulting (altom.fi). With more than ten years of experience in software testing, she is particularly interested in mobile testing and test automation with a special interest in mobile games. Ru is an instructor for the Black Box Software Testing series offered by Altom in collaboration with KFA. She is an organizer of the Test Lab, an open-space concept organized at testing conferences where participants practice testing and learn from their peers.

'HWHFWLRQ�7KHRU\�$SSOLHG�WR�)LQGLQJ�DQG�)L[LQJ�'HIHFWV

5X�&LQGUHD���$OWRP�&RQVXOWLQJ

$ERXW�PH

Ɣ IURP�5RPDQLD��QRZ�OLYLQJ�LQ�)LQODQG

Ɣ VWDUWHG�DV�D�WHVWHU�LQ�����

Ɣ SDUWQHU�DQG�VRIWZDUH�WHVWHU�DW�$OWRP�VLQFH�����

Ɣ %%67�,QVWUXFWRU��ZRUNLQJ�ZLWK�&HP�.DQHU�DQG�%HFN\�)LHGOHU�RIIHULQJ�%%67�FODVVHV�RQOLQH

Ɣ LQWR�PRELOH�WHVWLQJ��PRELOH�DXWRPDWLRQ�DQG�PRELOH�DSS�GHYHORSPHQW

#UXBDOWRP

6SHDN�(DV\KWWS���VSHDNLQJ�HDV\�FRP

7KDQNV�WR�0DDUHW�3\K¦M¦UYL��$QQH�0DULH�&KDUUHWW�DQG�)LRQD�&KDUOHV�IRU�VXSSRUW�DQG�PHQWRULQJ

7KH�VWRU\�RI�WKH�ZRUVW�EXJ�,�HYHU�IRXQG

ƌ,I�\RX�FDQ�UHSURGXFH�LW��WKH�GHYHORSHU�LV�ILUHG��,I�\RX�FDQƊW��\RXƊUH�ILUHG�ƍ

$�ORW�RI�LQWHUHVWLQJ�FRQGLWLRQV�IRU�WKH�EXJ

Ɣ ,�ZDV�D�MXQLRU�WHVWHU

Ɣ &RPSOH[�HQYLURQPHQW�ZLWK�PDQ\�GLIIHUHQW�([FKDQJH�VHUYHUV

Ɣ 0RVW�RI�WKH�FRPSRQHQWV�ZHUH�VLPXODWHG

Ɣ &RPPXQLFDWLRQ�YLD�EXJ�GDWDEDVH

Ɣ %XJ�ZDV�KDUG�WR�UHSURGXFH

Ɣ 6XEFRQWUDFWLQJ�DJUHHPHQW���EXJV�RQ�SURYLGHUƊV�WLPH�DQG�H[SHQVHV

Ɣ 'HYHORSHU�GLGQƊW�KDYH�WLPH

Ɣ &(2�GLGQƊW�NQRZ���GLGQƊW�XQGHUVWDQG�WKH�UHDO�SUREOHP

Ɣ 'HYHORSHUV�ZHUH�WDXJKW�WKDW�WHVWHUV�DUH�QRW�WR�EH�WUXVWHG

Ɣ 'HYHORSHUVƊ�WLPH�FRQVLGHUHG�PRUH�YDOXDEOH�WKDQ�WHVWHUVƊ�WLPH

Ɣ 'HYHORSHUV�QRW�HQFRXUDJHG�WR�LQYHVWLJDWH�GLIILFXOW�EXJV

,�GHFLGHG�WR�

䚉 UHSRUW�WKH�EXJ

䚉 QRW�WDON�WR�WKH�GHYHORSHU�GLUHFWO\�EXW�FRPPXQLFDWH�YLD�WKH�EXJ�GDWDEDVH

䚉 QRW�LQVLVW�WKDW�WKH�EXJ�VKRXOG�EH�IL[HG

䚉 QRW�WR�VSHQG�PRUH�WLPH�LQYHVWLJDWLQJ�LW

7KH�'HYHORSHU�GHFLGHG�WR�

䚉 QRW�WDNH�WKH�EXJ�VHULRXVO\

䚉 QRW�VSHQG�PRUH�WLPH�LQYHVWLJDWLQJ�LW

7KH�&(2�GHFLGHG�WR�

䚉 QRW�IL[�WKH�EXJ

䚉 QRW�DVN�DERXW�LWV�LPSOLFDWLRQV

䚉 QRW�WUXVW�PH

$�ORW�RI�GHFLVLRQV�ZHUH�LQYROYHG�LQ�KDQGOLQJ�WKH�EXJ

7KHVH�GHFLVLRQV�ZHUH�PDGH�XQGHU�FRQGLWLRQV�RI�XQFHUWDLQW\

7KHVH�GHFLVLRQV�ZHUH�VXEMHFW�WR�RXU�RZQ�ELDVHV�

'HWHFWLRQ�7KHRU\

Ɣ XVHG�ZKHQ�SV\FKRORJLVWV�ZDQW�WR�PHDVXUH�WKH�ZD\�ZH�PDNH�GHFLVLRQV�XQGHU�FRQGLWLRQV�RI�XQFHUWDLQW\

Ɣ GHFLVLRQ�PDNHU�LV�127�D�SDVVLYH�UHFHLYHU�RI�LQIRUPDWLRQ��EXW�PDNHV�GLIILFXOW�SHUFHSWXDO�MXGJPHQWV�

$Q�([DPSOH��'ULYLQJ�LQ�IRJJ\�FRQGLWLRQV

'HWHFWLRQ�7KHRU\���'ULYLQJ�LQ�IRJJ\�FRQGLWLRQV

Ɣ ZH�KDYH�WR�GHFLGH�KRZ�IDU�DZD\�IURP�XV�DQ�REMHFW�LV

Ɣ EDVHG�RQ�YLVXDO�VWLPXOXV�LPSDLUHG�E\�WKH�IRJ� !�FRQGLWLRQV�RI�XQFHUWDLQW\

'ULYLQJ�LQ�IRJJ\�FRQGLWLRQV 'ULYHU�GRHVQƉW�EUDNH 'ULYHU�EUDNHV

5HLQGHHU�RQ�WKH�VLGH�RI�WKH�URDG 0,66 +,7

1RWKLQJ�RQ�WKH�VLGH�RI�WKH�URDG &255(&7�5(-(&7,21 )$/6(�$/$50

'HWHFWLRQ�7KHRU\���'ULYLQJ�LQ�IRJJ\�FRQGLWLRQV

$SSO\LQJ�'HWHFWLRQ�7KHRU\�WR�)LQGLQJ�'HIHFWV

'HWHFWLRQ�7KHRU\���)LQGLQJ�'HIHFWV

Ɣ ZH�KDYH�WR�GHFLGH�LI�D�FHUWDLQ�EHKDYLRU�LV�D�EXJ�RU�QRW

Ɣ EDVHG�RQ�RXU�XQGHUVWDQGLQJ�RI�KRZ�WKH�SURGXFW�VKRXOG�ZRUN�DQG�EDVHG�RQ�WKH�VSHFLILFDWLRQV�ZH�DUH�DZDUH�RI

Ɣ EHFDXVH�ZH�PLJKW�QRW�QRWLFH�D�FHUWDLQ�EHKDYLRU�RU�PLJKW�QRW�XQGHUVWDQG�KRZ�WKH�SURGXFW�VKRXOG�ZRUN��ZH�PLJKW�WKLQN�WKH�SURGXFW�ZRUNV�DV�H[SHFWHG�ZKHQ�D�EXJ�RFFXUUHG�

)LQGLQJ�'HIHFWV 7HVWHU�GRHVQƉW�UHSRUW�D�SUREOHP 7HVWHU�UHSRUWV�GHIHFW

0LVEHKDYLRU�LQ�WKH�SURGXFW 0,66 +,7

%HKDYLRU�DFFRUGLQJ�WR�

VSHFV�H[SHFWDWLRQV

&255(&7�5(-(&7,21 )$/6(�$/$50

'HWHFWLRQ�7KHRU\���)LQGLQJ�'HIHFWV

$SSO\LQJ�'HWHFWLRQ�7KHRU\�WR�)L[LQJ�'HIHFWV

'HWHFWLRQ�7KHRU\���)L[LQJ�'HIHFWV

Ɣ ZH�KDYH�WR�GHFLGH�LI�D�FHUWDLQ�EXJ�VKRXOG�EH�IL[HG�RU�QRW

Ɣ EDVHG�RQ�RXU�XQGHUVWDQGLQJ�RI�KRZ�WKLV�EXJ�ZLOO�LPSDFW�WKH�SURGXFW

Ɣ EHFDXVH�ZH�PLJKW�QRW�XQGHUVWDQG�WKH�IXOO�LPSOLFDWLRQV�RU�FRQVHTXHQFHV�RI�D�GHIHFW��ZH�PLJKW�WKLQN�ZH�GRQƊW�QHHG�WR�IL[�WKH�GHIHFW�ULJKW�DZD\

)L[LQJ�'HIHFWV 'HFLVLRQ�WR�127�IL[�WKH�GHIHFW�LV�PDGH 'HFLVLRQ�WR�IL[�LW�LV�PDGH

'HIHFW�KDV�KXJH�LPSDFW�WR�PRVW�RI�WKH�FXVWRPHUV

0,66 +,7

'HIHFW�LV�LQGHHG�PLQRU�DQG�QRW�DW�

DOO�XUJHQW&255(&7�5(-(&7,21 )$/6(�$/$50

'HWHFWLRQ�7KHRU\���)L[LQJ�'HIHFWV

%DFN�WR�WKH�ZRUVW�EXJ�,�

HYHU�IRXQG���

Ɣ ,�IRXQG�DQG�UHSRUWHG�D�GHIHFW��D�UHDO�SUREOHP�� !�+,7

Ɣ 7KH�&(2�WKRXJKW�LW�ZDV�D�)$/6(�$/$50�

Ɣ 7KH�&(2�GHFLGHG�WR�QRW�KDYH�WKH�EXJ�IL[HG� !�0,66

$�ELW�RQ�ELDV

:H�FDQƊW�FRQWURO�RXU�ELDV��ZH�GRQƊW�NQRZ�KRZ�WR�127�EH�ELDVHG

HJ��([SHULPHQWHU�%LDV

%XW�ZH�FDQ�ELDV�RXUVHOYHV��RU�RWKHUV��LQ�D�GLIIHUHQW�GLUHFWLRQ���EDODQFH�WKH�VFDOH�GLIIHUHQWO\

$�ELW�PRUH�RQ�ELDV

0DLQ�ELDVLQJ�YDULDEOHV

Ɣ 0RWLYDWLRQ���3UHIHUUHG�UHVXOW

Ɣ 3HUFHLYHG�3UREDELOLW\

Ɣ ([SHFWHG�FRQVHTXHQFHV

Ɣ 3HUFHLYHG�LPSRUWDQFH�RI�WKH�WDVN

KWWSV���ZZZ�IOLFNU�FRP�SKRWRV���������#1��������������

:H�FDQ�PDQLSXODWH�

ELDV

81&216&,286�35()(55('�5(68/7��ƌ,�+23(�7+(5(�$5(�

12�5(,1'((5ƍ

(;3(5,0(17(5�%,$6�

KWWSV���ZZZ�IOLFNU�FRP�SKRWRV���������#1��������������

:H�FDQ�PDQLSXODWH�

ELDV&+$1*(�3(5&(,9('�352%$%,/,7<���025(�

ƌ+,76ƍ

KWWSV���ZZZ�IOLFNU�FRP�SKRWRV���������#1��������������

:H�FDQ�PDQLSXODWH�

ELDV&+$1*(�3(5&(,9('�352%$%,/,7<���025(�

ƌ+,76ƍ

025(�)$/6(�$/$506�$6�:(//�

KWWSV���ZZZ�IOLFNU�FRP�SKRWRV���������#1��������������

:H�FDQ�PDQLSXODWH�

ELDV&+$1*(�3(5&(,9('�

352%$%,/,7<���/(66�)$/6(�$/$506

KWWSV���ZZZ�IOLFNU�FRP�SKRWRV���������#1��������������

:H�FDQ�PDQLSXODWH�

ELDV&+$1*(�3(5&(,9('�

352%$%,/,7<��/(66�)$/6(�$/$506

025(�Ƌ0,66(6ƌ�$6�:(//�

KWWSV���ZZZ�IOLFNU�FRP�SKRWRV���������#1��������������

:H�FDQ�PDQLSXODWH�

ELDV&+$1*(�3(5&(,9('�

,03$&7�&216(48(1&(6

$*$,1��025(�)$/6(�$/$506

%LDV�SHRSOH�ZKR�

ILQGEXJV

025(�%8*�5(32576�

Ɣ HQFRXUDJH�UHSRUWLQJ�DQ\WKLQJ�WKDW�ORRNV�VWUDQJHƔ SUDLVH�JRRG�ZRUNƔ FRQWHVWVƔ KHOS�WHVWHUV�XQGHUVWDQG�WKH�SRVVLEOH�

FRQVHTXHQFHV�FRVWV�RI�D�GHIHFWƔ SUDLVH�ZRUN�RQ�EXJV�WKDW�DUH�YDOXDEOH�HYHQ�LI�GHIHUUHG

DQG�NHHS�LQ�PLQG�WKDW�ELDVLQJ�WRZDUGV�PRUH�EXJ�UHSRUWV�DOVR�PHDQV�PRUH�GHIHUUHG�QRQ�YDOLG�RQHV

%LDV�SHRSOH�ZKR�

ILQGEXJV

025(�9$/,'�5(32576�/(66�)$/6(�$/$506�

Ɣ LPSRUWDQW�GXULQJ�GLIIHUHQW�SKDVHV�LQ�WKH�SURMHFW��HDUO\�RQ��!�UHSRUW�HYHU\WKLQJ�

Ɣ HQFRXUDJH�GLUHFW�LQIRUPDO�FRPPXQLFDWLRQ��ƉPHQWLRQ�LQ�SDVVLQJƊ�

Ɣ H[SODLQ�ZK\Ɣ WUDLQ�SHRSOH��JLYH�H[DPSOHVƔ DFFHSW�\RX�PLJKW�KDYH�PRUH�PLVVHG�EXJV�DV�ZHOO

25«��OLNH�LQ�P\�VWRU\�

%LDV�SHRSOH�ZKR�

ILQGEXJV

025(�9$/,'�5(32576�/(66�)$/6(�$/$506���7+(�5,6.<�:$<�

Ɣ HYDOXDWH�WHVWHUV�EDVHG�RQ�KRZ�PDQ\�LQYDOLG�EXJV�WKH\�UHSRUW

Ɣ HYDOXDWH�GHYHORSHUV�DQG�WHVWHUV�EDVHG�RQ�DPRXQW�RI�WLPH�VSHQW�LQYHVWLJDWLQJ�LVVXHV�WKDW�PLJKW�QRW�EH�YDOLG�DIWHU�DOO

Ɣ XVH�D�WHVWHUƊV�EXJ�UHSRUWV�WR�SRLQW�RXW�PLVWDNHVƔ GLVPLVV�EXJV�UHSRUWHG�E\�MXQLRUVƔ QRW�PRQLWRU�ODQJXDJH�WRQH�RI�EXJ�UHSRUWV�DQG�FRPPHQWV�

WR�WKHP

+RZ�FRXOG�,�KDYH�KDQGOHG�WKH�VLWXDWLRQ�EHWWHU�LQ�WKDW�VWRU\"

%LDV�SHRSOH�ZKR�GHFLGH�RQ�IL[LQJ�EXJV

$6�$�7(67(5��<28�&$1�

Ɣ EXLOG�\RXU�FUHGLELOLW\�DQG�NQRZ�ZKDW�\RXU�UHSXWDWLRQ�LV

Ɣ ZRUN�RQ�WKH�TXDOLW\�RI�\RXU�EXJ�UHSRUWV

Ɣ OHDUQ�WR�HIIHFWLYHO\�LQYHVWLJDWH�KDUG�WR�UHSURGXFH�EXJV

$6�$�7(67(5��<28�&$1�

Ɣ ZRUN�RQ�WKH�WRQH�RI�\RXU�EXJ�UHSRUWV

Ɣ DFFXUDWHO\�HVWLPDWH�VHYHULW\�LPSDFW�RI�EXJV

Ɣ DYRLG�H[DJJHUDWLRQ�

%LDV�SHRSOH�ZKR�GHFLGH�RQ�IL[LQJ�EXJV

$6�$�7(67(5��<28�&$1�

Ɣ OHDUQ�ZKHQ�WR�JLYH�XS��DQG�ZKHQ�QRW�

Ɣ OHDUQ�ZKR�FDQ�KHOS�\RX�RXW���PDNH�ƌIULHQGV�ZLWK�SLFN�XS�WUXFNVƍ�

%LDV�SHRSOH�ZKR�GHFLGH�RQ�IL[LQJ�EXJV

$6�$�7(67(5��<28�&$1�

ƌ7UHDW�WKH�SURJUDPPHUV�DQG�RWKHU�VWDNHKROGHUV�ZLWK�FRXUWHV\�DQG�UHVSHFW��HYHQ���HVSHFLDOO\���RQHV�ZKR�GRQƊW�GHVHUYH�LW�ƍ�

�&HP�.DQHU��%XJ�$GYRFDF\�&RXUVH��/HVVRQ����

%LDV�SHRSOH�ZKR�GHFLGH�RQ�IL[LQJ�EXJV

7KDQN�\RX�#UXBDOWRP

UX�FLQGUHD#DOWRP�IL

48(67,216"

top related