there will be bugs! · 2014-11-21 · simplify, simplify, simplify kernighan: "debugging is...
TRANSCRIPT
"Will there be bugs"?YES!!!
with probability above 0.9999... Murphy's Law fully applies (& then some:-)
22
"When will there be bugs"?when you're pushing the envelope
new or new-to-you algorithms, fields, libraries, frameworks, ... ...there are new traps waiting for you
when you're doing humdrum, routine development or maintenance...
...what you've done 1,000 times before...
...your attention level's likely not 100%! ...and, in all cases in-between:-)
33
The 1 exception in my lifemy very first program: 1 run, bug-free 1974: Fortran, conditional probabilities of suits in bridge, mainframe 3 HW majors, bridge enthusiasts (key!), no programming courses (NP), humble (!) punched cards,"big brain" mystique (helped!) "invented" code-reviewing, pair-prog x 1.5 had PCs been easily around (a bit later)...
...bugs would have abounded (later did!-) ...never ever happened again in my life!
44
Fast Forward 40 years...downside: we only got 1 run so couldn't possibly have run tests
not that we'd ever heard of testing!-) nowadays, tests must be at the heart of bug avoidance, discovery, and fixing ...as they should have been in the '70s!-)
Knuth, 1977: "Beware of bugs in the above code; I have only proved it correct, not tested it"...!-)
but code reviews & pairing still help!
55
Where do bugs like to hide?Well, anywhere, actually!-)
66
Common bug-hiding places"advanced" stuff, or where you're "clever"
key fix: *simplify* (+, unit-test!) where you did not fully understand the problem, architecture, platform
ditto +: acceptance tests, code reviews boilerplate/duplicate code
Don't Repeat Yourself (DRY) rarely executed code (error handling, ...)
key fix: unit-test w/mocking for errors previously-buggy code ("regressions")
a special case of TDD
77
Simplify, simplify, simplifyKernighan: "Debugging is twice as hard as writing a program in the first place. If you're as clever as you can be when you write it, how will you ever debug it?" break up long, complicated expressions (intermediate results -> local variables) beware complex decision chains
and nested loops / recursion, esp. with many conditional break/continue stmts
regular expressions MUST be tested a LOT ( also see: regex101.com )
88
Don't Repeat Yourself (DRY)Hunt and Thomas (1999): "Every piece of knowledge must have a single, unambiguous, authoritative representation in a system". AKA Once and Only Once (focus on code)
copy-and-paste coding is a main anti-pattern breaking DRY / OaOO duplicate code is the worst code smell
everything's changing all the time: with duplicates, you'll miss some needed change abstract what varies, merge what doesn't
99
Before DRY, it's WET...:if foo(bar, baz):! return bar!else:! return baz!! ...!!if foo(zip, zap):! return zip!else:! return zap + 1
1010
...let's DRY it up!def picker(a, b, c=None):! if foo(a, b):! return a! else:! return b if c is None else b + c! ...!return picker(foo, bar)! ...!return picker(zip, zap, 1)!
1111
Rarely executed codehandling weird errors, corner cases unittest.mock (3.3+; backports aplenty) with mock.patch(...) as x:!
to locally replace ... with a Mock obj x set return_value, side_effect, ...
after, may assert about x's calls &c BEWARE:
all attributes and methods auto-appear -> high danger of typoes!
use auto_spec to reduce the danger
1212
A mock.patch exampledef foo():! try: never.fails()! except OopsItDid as e:! logging.warm('Failed: %s', e)! raise! ...!with mock.patch('never.fails'‡) as nf:! nf.side_effect = OopsItDid! with self.assertRaises(OopsItDid):! module_i_am_testing.foo()!
‡: add autospec=True &c
1313
Why do bugs happen?Your brain tricks you!
Perception: See what you expect to see Attention lapses Overconfidence Confirmation bias
You've written it: it's YOUR code, it's YOUR baby, you're really proud of it
bugs love to hide there, as `you won't see them` -- you're "too close" to see clearly
1414
Remedies for your “tricksy brain”egoless programming more eyeballs
open source pair programming code reviews
testing `lint` and similar static-analysis tools
1515
Egoless ProgrammingJerry Weinberg, "The Psychology of Computer Programming", 1971 1.Understand and accept that you will
make mistakes; find them early! 2. You are not your code; reviews are to
find problems, and find them they will -- don't take it personally!
... 10. Critique code instead of people – be
kind to the coder, not to the code
1616
Eyeballs: open sourceEric Raymond's "Linus Law" (1997): "given enough eyeballs, all bugs are shallow"
"Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix will be obvious to someone."
Glass "Facts and Fallacies about Software Engineering" (2003): "it's a fallacy"
"the rate at which additional bugs are uncovered does not scale linearly with the number of reviewers"
`goto fail`, `heartbleed`; see Mike Bland's http://martinfowler.com/articles/testing-culture.html
1717
Eyeballs: pair programmingthe "driver" codes, focuses on "tactical" aspects of completing the current task the "observer" reviews-as-it-goes, ponders "strategic" issues, plays "safety net"
frequent role switches; talking through it many empirical studies and meta-research confirm: higher quality, faster time, but larger effort, compared with `solo` progr.
effort estimation here is biased-high e.g: P.P produces a higher `bus number` w/o extra study/analysis effort!
1818
Eyeballs: code reviewstraditional "high ceremony" `Fagan Inspections` -- multiple meetings, printed code, thorough line-by-line critique, ...
high cost, very slow, high efficacy may "fail to see the forest for the trees", "bikeshedding" also a risk
lightweight, informal walkthroughs and critiques - much faster, also effective
especially with the right tool! complementary, not alternative, to pair programming and testing
1919
TestingJacob Kaplan-Moss: "code without tests is broken by design":-) informal "just run it and see if it breaks"
quite ineffective, esp. by original author automated light-weight "unit tests"
must run fast so you can and will keep re-testing even on minor changes great at avoiding unit-specific issues
automated middle-weight "integration tests" complementary, not alternative, to u-t's focus on interaction between subsystems
formal heavy-weight "acceptance tests"/QA2020
Testing vs EyeballsTesting: objectively shows bugs' presence
can never conclusively show absence:-) automated -> re-run all the time
Eyeballs: complementary, not alternative pick up issues testing can never reveal
lack of clarity, bad naming, complexity not automated -> consume human time
A third leg for the anti-bug "stool": lint automated, fast uniform style -> more productive eyeballs
2121
Anti-bug toolsmost important is source code control (svn, hg, git, ...) -- trace what changed and when
beware subtle merges, cherrypicks, ... next, a testing framework (unittest &c) a bug tracker (ideally integrating w/SCC) a code review tool (ditto)
2222
“What about debuggers?”only mild importance for fighting bugs
if the code's so complicated that you have to follow step by step, simplify it
can help as "learning device" unfamiliar code, library, framework
2323
Beware tool-itis!a geek's natural attitude: "if I have the right tools, all bugs will flee in terror"
they won't flee: bugs are courageous!-) tools, at best, help: they don't solve bugs tools can in fact hurt, by distracting you
hours spent in front of a cool debugger ...yak-shaving (tracing just-fine code)...
MUCH more important than tools' details: attitude, skill, care, focus on team&user and esp: good practice (light process)
2424
And yet, tools are cool:-)...but, excessive power can hurt the unwise
debugger to interactively examine values: log them instead! (w/logging.debug)
then code a script to sanity-checks logs big difference: it's automated!
SCC allows fancy integrates, merges, cherrypicks, and generally funky graphs
very high risk of introducing bugs! keep SCC graphs clean and simple!
key idea: good enough IS good enough
2525
Testing frameworksstdlib unittest: a good starting point
+ many extensions: nose & plugins, &c coverage's important (figleaf's ok too:-)
use doctest only for examples in docs! it's designed for that & does it well
automated test runners / CI nosy, nosier, nosyd, PyZen; buildbot
mocks, fakes, stubs, spies, dummies, ... unittest.mock, but use with care!-)
specialized: web, fuzzing, GUI, acceptance...
2626
Web testingeither: simulate a browser
simple, fast -- but, no Javascript, CSS &c! best for unit-tests
or: automate a real browser most powerful, realistic, but, slower best for integration & acceptance tests
specific web frameworks may further help from google.appengine.ext import testbed!from django import test!...
2727
Good enough...pick one from each group
the standard-er, the better
2828
Example: web testingeither: simulate a browser
simple, fast -- but, no Javascript, CSS &c! best for unit-tests
or: automate a real browser most powerful, realistic, but, slower best for integration & acceptance tests
specific web frameworks may further help from google.appengine.ext import testbed!from django import test!...
2929
Linting &cLogilab's pylint
very powerful, configurable unused/unassigned variables too-long modules/functions style violations in naming, wspace, ...
pyflakes, pep8: limited, but fast Clone Digger
finds some "clones" (duplicate code) ...
3030
When to use a bug trackerAlways!-)
"bug tracker" (BT) is a misnomer...: track both bugs AND features
Integration w/SCC: any changeset must identify which BT entry it regards
ideally only 1: keep CSs small! Integration w/code review tool: code reviewer can follow the BT entry to find out WHY the CS under review exists BT entry points back to CSs fixing it
3131
The worst kind of bugrace conditions (& their evil cousins: deadlock, starvation, ...) alas, test don't help much (!); code reviews may if super-duper-hyper careful/thorough prevention is by far the best cure here
avoid shared-RW-memory concurrency message passing, shared-RO-mem OK
if you really can't, rigorously sequence lock acquisitions, a la Dijkstra
maybe a dedicated global-memory-changing thread w/Queue of changes
3232
Test-Driven DevelopmentAgain: Jerry Weinberg, "The Psychology of Computer Programming", 1971 `In program testing, the programmer who gets early "success" with his program is likely to stop testing too soon. One way to guard against this mistake is to prepare the tests in advance of testing and, if possible in advance of coding.` write the tests; see them fail; fix code to pass each test; check they succeed; refactor and check they still succeed.
3333
TDD: worth using? (1)(personal opinion here, but, based on substantial experience...) for adding features...: hmmm...
what exactly do you unit-test? proper test targets for most features, "user stories", are more suitable for acceptance/integration than unit tests
Behavior driven development (BDD): TDD variant based exactly on user stories
ideally written by users/PMs/&c great if you get such users/PMs/&c:-)
3434
TDD: worth using? (2)for fixing bugs...: absolutely YES!
you know exactly what to unit-test for: the very bug you're fixing!
first, you write the new tests ...and make sure all the new tests fail "reproduces the bug" in a known state
then, you fix the bug ...and make sure all the tests pass
the new tests stay in the test-suite insurance against future regressions!
3535
TDD: worth using? (3)for refactoring...: just *NOT APPLICABLE*!
you can NEVER safely refactor code not already well covered by good tests ensuring such coverage is not "the start of the refactoring",
it's a *PRE-REQUISITE* of it! Michael Feathers, "Working Effectively with Legacy Code", 2002 -- http://www.objectmentor.com/resources/articles/WorkingEffectivelyWithLegacyCode.pdf
3636
Q & Ahttp://www.aleax.it/bayp14_twbb.pdf!
37
? !37