Download - Lecture 16a Walkthroughs.ppt 0
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
1/59
Software Reviews,
Walkthroughs, Inspections, and
AuditsBetter to find an error twice than
never to find it at all.
[Freedman/Weinberg !" Freedman,
#$%$, &$'$ Weinberg, ()andbook of
0pdate terminolog1
according
&ive references
to 234eil
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
2/59
Wh1 #o We )ave Formal
*echnical Reviews5
$ *o err is human$6$ 7ots of errors escape the originator more
easil1 than an1one else$
8$ Reviews are educational$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
3/59
%urpose9 %rovide
: visibilit1 into state of pro;ect
: opportunities for personnel uac1 of pro;ect
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
4/59
0se the power of a team to ?
: %oint out needed improvements of a
product$
: +onfirm the parts in which improvement is
not needed or desired$
: 'ake technical work more uniform and
predictable in >ualit1, which makes it moremanageable$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
5/59
*echnical review is different
from other reviews: For e@ample, budget and management
reviews
: *echnical review answers >uestion9 will
this product do the ;ob it3s supposed to do
: If answer is noB, then
Cno schedule is on time, and
Cno cost is cheap enough$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
6/59
When5
: Single review that occurs on a particular date
: Single review that occurs in response to a
particular condition: 'ultiple reviews that occur periodicall19 e$g$,
monthl1
: 'ultiple reviews that occur in response to aparticular condition, e$g$, code review forsubs1stem
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
7/59
Denefits
: !E reduction in errors in products
: !.-!G cost reduction
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
8/59
Wh1 not ;ust pass it around and
have reviewers sign off ?
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
9/59
Wh1 not ;ust pass it around and
have reviewers sign off ?
: +omments are too large: +omments are unread
: Reviewers don3t have to take responsibilit1
for the content$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
10/59
Re>uirements
: *eam participation
: #ocumented procedures
: #ocumented results
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
11/59
*1pes of Reviews
: 'anagement Review
: *echnical Review
: Walkthrough
: Inspection
: Audit
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
12/59
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
13/59
'anagement Review
: 2b;ectives9
C Inform management of pro;ect status
C Resolve higher.level issues
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
14/59
*1pes of Reviews
: 'anagement Review
: *echnical Review
CFocus on specification and design
C#etermine whether products meet specs
: Walkthrough
: Inspection
: Audit
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
15/59
Scenario for a t1pical review
: 7eader9 Welcome$ We are here to review E for the
purpose of $
: Hach person is introduced$: *he recorder is introduced$
: An agenda is presented and the review process
e@plained$: *he list of material in the review packet is stated$
: *he review begins?
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
16/59
Review
: Agenda set b1 leader, agreed to b1
committee
: Review proceeds line.b1.line$
: Reviewers bring up issues as the1 encounter
them$
: Issues are added to an Issues 7ist$
: Agenda changes as issues are encountered$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
17/59
Round Robin Review
: Force ever1 member to participate$
: Hach person takes the lead for a section9
Cparagraph,
Cline of code,
Cfunction,
Ctest case
: &ood wa1 to get involvement from team$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
18/59
*1pes of Reviews
: 'anagement Review
: *echnical Review
: WalkthroughC 0sed to find anomalies and evaluate conformance
C *1picall1 used for source code
C Hffective form of education
: Inspection
: Audit
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
19/59
Walkthroughs
: Producerguides review
: Step b1 step presentation of product9
C+ode, design, report, test cases ?: +an cover lots of material
: +an have more people attend, less prepared
: 'ore work for presenter
: 'a1 be difficult to control interactions
C*oo man1 interruptions
CAlwa1s hard to have a producer present
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
20/59
Walkthroughs
: %articipant should spend hour preparing
for each hour in meeting
: Should be scheduled to be brief
: Should onl1 review completed code or
document
: Should re>uire participants to sign report
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
21/59
*1pes of Reviews
: 'anagement Review
: *echnical Review
: Walkthrough
: Inspection
C'ore formal t1pe of walkthrough
C0sed to identif1 specific errors or problems
: Audit
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
22/59
Inspections
: Rapid evaluation of material with specific
aspect in mind
: +onfine attention to one aspect onl1
CHg assume details are correct, but look for
missing items at a global level
CHg look for occurrences of a particular kind ofbug such as buffer overflow$
: Selection of aspects is ke1 point$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
23/59
*1pes of Reviews
: 'anagement Review
: *echnical Review
: Walkthrough
: Inspection
:AuditCIndependent evaluation of process or product
CHnsures that the process is being followed
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
24/59
Review *eams
: %ermanent or semi.permanent teams
: IJKJ or LA teams
: &et to be ver1 good at reviews
: 4eed to have some wa1 of reviewing the
reviewers$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
25/59
*he team: 7eader
CMob9 obtain a good review or report wh1 it wasnot possible
: Reflects the >ualit1 of the review process,not the >ualit1 of the product
: 'ust be accurate
: Recorder
C%rovide info for accurate report of review
CShort, public notesC+apture essence of each issue
C'ust ensure group has reached conclusion
C#on3t video tape
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
26/59
*asks of leader
: 'onitor preparedness of team members
: Set pace
: Neep meeting on track: %oll members to reach consensus
: Hnsure participation
: )as right to terminate review if it is unproductiveC#isagreements
CFatal flaws
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
27/59
+onsensus
: #ecision of the group is e>ual to the most
severe opinion of the group members$
: De conservative$
: Accept the doubts of the most doubting
member$
: 4o I told 1ou so, but I got outvotedB$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
28/59
Reviewers
: 'ust be >ualified to contribute
: 'ust be unbiased
: #on3t invite management if it causes
conflict
C*he point is to review the pro;ect, not the
producers$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
29/59
Rules for Reviewers
I$ De prepared
II$ Hvaluate product, not people$
III$ Watch 1our language$
IJ$ 2ne positive, one negative$
J$ Raise issues, don3t resolve them$JI$ Record all issues in public$
JII$ Stick to technical issues$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
30/59
Rules for Reviewers
: If 1ou find something, assume it3s a mistake$*hese are 1our peers, not 1our enem1$ Remember
people are involved
: Avoid wh1 did 1ou ?B wh1 didn3t 1ou ?B sa1
instead I don3t understand ?BC Wh1 did 1ou set the upper bound to ! here5BC I don3t understand wh1 the upper bound is ! here$BC seems trivial, but it3s not$
: #o not get bogged down in st1le issues$ Fore@ample, if efficienc1 isn3t an issue, then don3tmake it one$
: If it makes things less maintainable, that3s an issue$
: If there are standards, then either stick to thestandard, or dispose of the standard$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
31/59
4umber of Reviewers
: Hnsure material is covered
: )ave enough
: #on3t have too man1
: 8.O is good
C+ount participants onl1
C89 ensures material is understood
: 2utsiders can be good$ 'ust be unbiased
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
32/59
*ime
: At most 6 hours at a time
: +ome back again if necessar1
: #epends on
C+omple@it1
CSiPe of pro;ect
C+loseness to completion
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
33/59
Hver1one must prepareQ
: Review packet
: Hver1thing relevant to making correct
;udgment9 code, specs, test data, results,standards, designs, ?
: -!G of failures stem from unprepared teams
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
34/59
*actics for participation: #evil3s Advocate
C2ne person makes strongest case against aproduct
CMob is to find fault
C4eeds to be an actor: #ebugging
C%roducers leave known bugs in
CLualit1 of review depends on how man1 found
: Alarms
C*ime each person3s contribution, cut off
: Stand.up Reviews
C*alk as long as 1ou stand on one leg
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
35/59
Report
: &oal9 #oes the product do the ;ob its supposed todo5
C+omplete$
C+orrect$
C#ependable as basis for other work
C'easurable for purposes of tracking$
: What was reviewed5: Signatures of reviewers$
: 7eader and recorder$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
36/59
4otes
: Re>uires learning9 to review and to write
: 4eed to allocate time
: 4ot a form of pro;ect management
CDut can provide information for tracking
: Review earl1 and often
CDut not too often
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
37/59
4eed for Reviews
: Reviews help organiPation deliver >ualit1product
CH@amine individual parts earl1 in processCIdentif1 problems spanning multiple items or
phases
: Reviews help identif1 problems earl1
: Reviews help build image of the productand the vision of the product in the minds ofthe team members
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
38/59
+ost of Software Reviews
: 0p.front cost forC *raining
C Staff preparation
C Review +onduct
: +ode review ma1 be -.G of total cost of pro;ect
: 2ffset b1 savings9 reduced rework later in pro;ect
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
39/59
Writing Issues
: &roups of 8
: I3ll give 1ou an issue from an issue list,
: ou tell me what3s wrong with it$
: Issues
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
40/59
Writing Issues
Some of the e@planations of user commands
were misinterpreted b1 members of the
review committee$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
41/59
+omments on Issues
4ot specific enough$ Whiche@planations were
misinterpreted5 #on3t make the producers
guess, or the1 ma1 change the wrong thing$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
42/59
Writing Issues 6A ma@imum of ! values ma1 be specified
even though none of the standard s1stem
ADH4# codes has been made not eligible$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
43/59
+omments on Issues 6
In writing, clarit1 isn3t the most important
thing9 it3s the onlything$ If 1ou don3t want
to be not understood, don3t never use nodouble negatives$
Another thingC what is the issue an1wa15 Is
the ma@imum too low5 *oo high5 2r is itthat some of the codes should have been
made not eligible5 2r eligible5 De direct$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
44/59
Writing Issues 8
*he referenced table of constant values was
not part of the review packet$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
45/59
+omments on Issues 8
Alwa1s give the most direct reference
available$ *here could be more than one
table, now or in the future$ Wh1 make theproducers search5
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
46/59
Writing Issues
*he method used for maintaining the message
>ueue seems to solve a severe performanceproblem we3ve been having with the
production version of the RN s1stem$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
47/59
+omments on Issues
*his is an interesting observation, clearl1 and
directl1 stated$ Dut what does it have to do
with the product under review5 It might beworth millions, but it doesn3t belong on the
Issues 7ist for this product$ It should go on
the Related issues list$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
48/59
'ore issues *he price/performance table should be sorted
using either Luicksort or Shell sort$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
49/59
+omments
Raise issues, but avoid all temptation to giveadvice in the Issues 7ist$ *he1 won3t be
welcome if the1 come in that form, so if1ou reall1 must give advice, find someunofficial vehicle$ *ake the producers tolunch, or out for a beer, before 1ou share
1our vast e@perience on matters of sorting$If 1our idea isn3t worth the price of a beer,wh1 not forget it5
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
50/59
'ore issues
*he three diagrams drawn b1 )arold 'itter
are not in the standard format uired for such diagrams in our
installation$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
51/59
+omments
*his point is nicel1 specific, but wh1 do we
have to mention poor )arold5 We are
reviewing the product, not the people$ Findanother wa1 to identif1 the diagrams and
leave people3s names out of the Issues 7ist$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
52/59
'ore issues O
*he committee was unable to understand the
significance of paragraph 8$$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
53/59
+omments O
4othing3s wrong with this one$ It3s specific aboutwhich paragraph is under discussion, and it sa1sthe committee doesn3t understand it$ Who can
argue with that5 et, surprisingl1, people seem tobe afraid to e@press issues this wa1 C We don3tunderstand ?B #on3t worr1 about being thoughtstupid$
If 1ou don3t understand it, it3s at least a potentialissue in documentation$ And besides, lack ofunderstanding ma1 mean there is somethingdreadfull1 wrong$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
54/59
et more issues -
*he bubble sort used for sorting the table of
price/performance figures is a stupid
approach if the table should grow an1
bigger than the present 6! entries$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
55/59
+omments -
*he word stupidB doesn3t add an1thing at all and
might antagoniPe the producers$ *ake it out$ *hen
tr1 to e@press factuall1 and >uantitativel1 wh1 the
method is inappropriate for larger tables$
And if the approach isstupid, what of it5 As Arthur
+$ +larke e@pressed it, It has 1et to be proved
that intelligence has an1 survival value$B
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
56/59
et more issues
If the D 4A'H option is not used, the
structuring of the structure operands must
be e>uivalent to the structuring of the
structures in the arra1s of structures$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
57/59
+omments
*his wasn3t reall1 taken from an Issues 7ist,
but from an old %7/I manual$ Still, I3ve read
real issues that were almost as obscure$ I3dadvise 1ou to follow the NISS a@iom
uestion$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
58/59
et more issues !
Frieda Sonntag has no computer science
background, and her e@perience with %7/Iis nil$ She should never have been
assigned the coding of this module$
-
8/13/2019 Lecture 16a Walkthroughs.ppt 0
59/59
+omments !
It3s non of the business of the review
committee who management has assigned
to particular ;obs$ *he committee3s businessis to review the product and tell what state
it3s in$ )ow it got to that state is another
issueTand not for the Issues 7ist$ Who isresponsible is even less of an issue, and
raising it is sure to be ineffective$