maybe we don’t have to test it

(Maybe We Don’t Have to)

Test It

Preparing For Houseguests(A Story)

TheProblem

11

1 1 100

001 00

Tester

11 1

00

01000

10 0

1 0

1 1001 100 00 000 1 11

11 1

Video of arrogant tester.

Routines are safe.

“If it exists, I want to test it”

James Bach

“If it exists, I want to test it. (The only exception is if I have

something more important to do.)”James Bach

11

1 1 100

001 001

1 10

00100

10 0

1 0

1 10

0

1 100 00 000 1 1

11

1

100

100

100

11

1 1 00

01 0010 0

1 101 1001

01

100

1

Patterns

1No repro steps.

Bug106621:DimMedia returns error intermittently.

How can I repro this?

Beats me.

I’ll play around in QA.

Have fun.

Um, no luck yet.

You’re wasting time.

Can’t you debug it or something?

No.

How will we know if it’s fixed before we

release to prod?

We won’t.

2Things that don’t go to

production.

Feature 106218:As a developer, I would like to convert TypeMock unit tests to Microsoft Fakes…

Feature 96120:

Create visualization of

service operation

dependencies from SM

data.

Feature 96190:

Create service

latency reports to

more easily

troubleshoot

problems

3Patches for critical

production problemsthat can’t get worse.

Bug 102927:

Users unable to rework today’s schedule with empty alternate.

Bug 103789:CopyEngine.ApplyVirtualCopyBySchedule causing perpetual LogDataMart timeout.

4Cosmetic changes with

timely test setup.

Feature 103540:

The word “unknown” is

spelled incorrectly on

the alt time constraint

validation message.

Would you bother to test this?

Yes No0

10

20

30

40

5Straight forward

configuration changes.

Feature 106863:

TVE DS: Disable SCTE

events for CNN & HLN

Bug 94337:SolverPlatform jobs throwing error while while trying to serialize parameter.

Maximum number of items that can be serialized or deserialized in an object graph is '65536'

Trust but

verify

6Too technical for a non-

programmer to test.

Feature 106221:

Janus TypeInitializer

should recover if Janus

Admin service is down

upon initial service call

after app pool reset.

7Non-tester on loan.

Feature 100251:Brand change notification cosmetic enhancements.

Feature 95947:

As a module user I

would like the module

search to be resizable so

I can see longer criteria.

Feature 94052:

As a member of the Proof

Group, I would like the

proof categories to be

more intuitive.

8Inadequate test data

or hardware.

Bug 105481:

CDC not properly re-

initializing after re-

initializing replication.

Bug 106271:Database script - users unable to open schedule for TNT, 1/11/2013 due to orphaned comment.

9Bugs not logged by

you.

Techniques

I’m not testing this because...

Are you crazy?

Only a fool would bother

testing this.

I’ll just mark it

“Tested” without doing

much.

An endorsement without careful thought or personal investment in the outcome.

1.Some degree of risk analysis took place.

2.The impact to production would be low.

RR 7/13/12: There is no feasible test

that can be done for this prior to

deployment due to replication not

happening in downstream

environments. The worst case if this

does not work, is CDC will still remain

broken. There is no risk of data loss.

Passing this work item along so it can

be deployed to production.

3 Questions to Start Asking:

1. Does this need to ever be tested?

2. Does it need to be tested by me?

3. Who cares if it doesn’t work?

“NOT testing” may still leave room for testing by

a broader definition

We may be…

• Regression testing/checking.

• Questioning our team.• Learning product

architecture.• Evaluating others’ tests.• Reviewing code.

Instead of saying “NOT testing”,

try saying“testing something more important”.

1. Spend more time on valuable testing.2. Look for “not-test” opportunities. 3. Use my 9 patterns and find your own.4. Introduce a code word.

Video of revised arrogant tester.