Transcript
  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    1/59

    Software Reviews,

    Walkthroughs, Inspections, and

    AuditsBetter to find an error twice than

    never to find it at all.

    [Freedman/Weinberg !" Freedman,

    #$%$, &$'$ Weinberg, ()andbook of

    0pdate terminolog1

    according

    &ive references

    to 234eil

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    2/59

    Wh1 #o We )ave Formal

    *echnical Reviews5

    $ *o err is human$6$ 7ots of errors escape the originator more

    easil1 than an1one else$

    8$ Reviews are educational$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    3/59

    %urpose9 %rovide

    : visibilit1 into state of pro;ect

    : opportunities for personnel uac1 of pro;ect

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    4/59

    0se the power of a team to ?

    : %oint out needed improvements of a

    product$

    : +onfirm the parts in which improvement is

    not needed or desired$

    : 'ake technical work more uniform and

    predictable in >ualit1, which makes it moremanageable$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    5/59

    *echnical review is different

    from other reviews: For e@ample, budget and management

    reviews

    : *echnical review answers >uestion9 will

    this product do the ;ob it3s supposed to do

    : If answer is noB, then

    Cno schedule is on time, and

    Cno cost is cheap enough$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    6/59

    When5

    : Single review that occurs on a particular date

    : Single review that occurs in response to a

    particular condition: 'ultiple reviews that occur periodicall19 e$g$,

    monthl1

    : 'ultiple reviews that occur in response to aparticular condition, e$g$, code review forsubs1stem

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    7/59

    Denefits

    : !E reduction in errors in products

    : !.-!G cost reduction

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    8/59

    Wh1 not ;ust pass it around and

    have reviewers sign off ?

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    9/59

    Wh1 not ;ust pass it around and

    have reviewers sign off ?

    : +omments are too large: +omments are unread

    : Reviewers don3t have to take responsibilit1

    for the content$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    10/59

    Re>uirements

    : *eam participation

    : #ocumented procedures

    : #ocumented results

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    11/59

    *1pes of Reviews

    : 'anagement Review

    : *echnical Review

    : Walkthrough

    : Inspection

    : Audit

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    12/59

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    13/59

    'anagement Review

    : 2b;ectives9

    C Inform management of pro;ect status

    C Resolve higher.level issues

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    14/59

    *1pes of Reviews

    : 'anagement Review

    : *echnical Review

    CFocus on specification and design

    C#etermine whether products meet specs

    : Walkthrough

    : Inspection

    : Audit

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    15/59

    Scenario for a t1pical review

    : 7eader9 Welcome$ We are here to review E for the

    purpose of $

    : Hach person is introduced$: *he recorder is introduced$

    : An agenda is presented and the review process

    e@plained$: *he list of material in the review packet is stated$

    : *he review begins?

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    16/59

    Review

    : Agenda set b1 leader, agreed to b1

    committee

    : Review proceeds line.b1.line$

    : Reviewers bring up issues as the1 encounter

    them$

    : Issues are added to an Issues 7ist$

    : Agenda changes as issues are encountered$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    17/59

    Round Robin Review

    : Force ever1 member to participate$

    : Hach person takes the lead for a section9

    Cparagraph,

    Cline of code,

    Cfunction,

    Ctest case

    : &ood wa1 to get involvement from team$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    18/59

    *1pes of Reviews

    : 'anagement Review

    : *echnical Review

    : WalkthroughC 0sed to find anomalies and evaluate conformance

    C *1picall1 used for source code

    C Hffective form of education

    : Inspection

    : Audit

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    19/59

    Walkthroughs

    : Producerguides review

    : Step b1 step presentation of product9

    C+ode, design, report, test cases ?: +an cover lots of material

    : +an have more people attend, less prepared

    : 'ore work for presenter

    : 'a1 be difficult to control interactions

    C*oo man1 interruptions

    CAlwa1s hard to have a producer present

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    20/59

    Walkthroughs

    : %articipant should spend hour preparing

    for each hour in meeting

    : Should be scheduled to be brief

    : Should onl1 review completed code or

    document

    : Should re>uire participants to sign report

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    21/59

    *1pes of Reviews

    : 'anagement Review

    : *echnical Review

    : Walkthrough

    : Inspection

    C'ore formal t1pe of walkthrough

    C0sed to identif1 specific errors or problems

    : Audit

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    22/59

    Inspections

    : Rapid evaluation of material with specific

    aspect in mind

    : +onfine attention to one aspect onl1

    CHg assume details are correct, but look for

    missing items at a global level

    CHg look for occurrences of a particular kind ofbug such as buffer overflow$

    : Selection of aspects is ke1 point$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    23/59

    *1pes of Reviews

    : 'anagement Review

    : *echnical Review

    : Walkthrough

    : Inspection

    :AuditCIndependent evaluation of process or product

    CHnsures that the process is being followed

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    24/59

    Review *eams

    : %ermanent or semi.permanent teams

    : IJKJ or LA teams

    : &et to be ver1 good at reviews

    : 4eed to have some wa1 of reviewing the

    reviewers$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    25/59

    *he team: 7eader

    CMob9 obtain a good review or report wh1 it wasnot possible

    : Reflects the >ualit1 of the review process,not the >ualit1 of the product

    : 'ust be accurate

    : Recorder

    C%rovide info for accurate report of review

    CShort, public notesC+apture essence of each issue

    C'ust ensure group has reached conclusion

    C#on3t video tape

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    26/59

    *asks of leader

    : 'onitor preparedness of team members

    : Set pace

    : Neep meeting on track: %oll members to reach consensus

    : Hnsure participation

    : )as right to terminate review if it is unproductiveC#isagreements

    CFatal flaws

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    27/59

    +onsensus

    : #ecision of the group is e>ual to the most

    severe opinion of the group members$

    : De conservative$

    : Accept the doubts of the most doubting

    member$

    : 4o I told 1ou so, but I got outvotedB$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    28/59

    Reviewers

    : 'ust be >ualified to contribute

    : 'ust be unbiased

    : #on3t invite management if it causes

    conflict

    C*he point is to review the pro;ect, not the

    producers$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    29/59

    Rules for Reviewers

    I$ De prepared

    II$ Hvaluate product, not people$

    III$ Watch 1our language$

    IJ$ 2ne positive, one negative$

    J$ Raise issues, don3t resolve them$JI$ Record all issues in public$

    JII$ Stick to technical issues$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    30/59

    Rules for Reviewers

    : If 1ou find something, assume it3s a mistake$*hese are 1our peers, not 1our enem1$ Remember

    people are involved

    : Avoid wh1 did 1ou ?B wh1 didn3t 1ou ?B sa1

    instead I don3t understand ?BC Wh1 did 1ou set the upper bound to ! here5BC I don3t understand wh1 the upper bound is ! here$BC seems trivial, but it3s not$

    : #o not get bogged down in st1le issues$ Fore@ample, if efficienc1 isn3t an issue, then don3tmake it one$

    : If it makes things less maintainable, that3s an issue$

    : If there are standards, then either stick to thestandard, or dispose of the standard$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    31/59

    4umber of Reviewers

    : Hnsure material is covered

    : )ave enough

    : #on3t have too man1

    : 8.O is good

    C+ount participants onl1

    C89 ensures material is understood

    : 2utsiders can be good$ 'ust be unbiased

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    32/59

    *ime

    : At most 6 hours at a time

    : +ome back again if necessar1

    : #epends on

    C+omple@it1

    CSiPe of pro;ect

    C+loseness to completion

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    33/59

    Hver1one must prepareQ

    : Review packet

    : Hver1thing relevant to making correct

    ;udgment9 code, specs, test data, results,standards, designs, ?

    : -!G of failures stem from unprepared teams

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    34/59

    *actics for participation: #evil3s Advocate

    C2ne person makes strongest case against aproduct

    CMob is to find fault

    C4eeds to be an actor: #ebugging

    C%roducers leave known bugs in

    CLualit1 of review depends on how man1 found

    : Alarms

    C*ime each person3s contribution, cut off

    : Stand.up Reviews

    C*alk as long as 1ou stand on one leg

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    35/59

    Report

    : &oal9 #oes the product do the ;ob its supposed todo5

    C+omplete$

    C+orrect$

    C#ependable as basis for other work

    C'easurable for purposes of tracking$

    : What was reviewed5: Signatures of reviewers$

    : 7eader and recorder$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    36/59

    4otes

    : Re>uires learning9 to review and to write

    : 4eed to allocate time

    : 4ot a form of pro;ect management

    CDut can provide information for tracking

    : Review earl1 and often

    CDut not too often

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    37/59

    4eed for Reviews

    : Reviews help organiPation deliver >ualit1product

    CH@amine individual parts earl1 in processCIdentif1 problems spanning multiple items or

    phases

    : Reviews help identif1 problems earl1

    : Reviews help build image of the productand the vision of the product in the minds ofthe team members

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    38/59

    +ost of Software Reviews

    : 0p.front cost forC *raining

    C Staff preparation

    C Review +onduct

    : +ode review ma1 be -.G of total cost of pro;ect

    : 2ffset b1 savings9 reduced rework later in pro;ect

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    39/59

    Writing Issues

    : &roups of 8

    : I3ll give 1ou an issue from an issue list,

    : ou tell me what3s wrong with it$

    : Issues

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    40/59

    Writing Issues

    Some of the e@planations of user commands

    were misinterpreted b1 members of the

    review committee$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    41/59

    +omments on Issues

    4ot specific enough$ Whiche@planations were

    misinterpreted5 #on3t make the producers

    guess, or the1 ma1 change the wrong thing$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    42/59

    Writing Issues 6A ma@imum of ! values ma1 be specified

    even though none of the standard s1stem

    ADH4# codes has been made not eligible$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    43/59

    +omments on Issues 6

    In writing, clarit1 isn3t the most important

    thing9 it3s the onlything$ If 1ou don3t want

    to be not understood, don3t never use nodouble negatives$

    Another thingC what is the issue an1wa15 Is

    the ma@imum too low5 *oo high5 2r is itthat some of the codes should have been

    made not eligible5 2r eligible5 De direct$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    44/59

    Writing Issues 8

    *he referenced table of constant values was

    not part of the review packet$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    45/59

    +omments on Issues 8

    Alwa1s give the most direct reference

    available$ *here could be more than one

    table, now or in the future$ Wh1 make theproducers search5

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    46/59

    Writing Issues

    *he method used for maintaining the message

    >ueue seems to solve a severe performanceproblem we3ve been having with the

    production version of the RN s1stem$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    47/59

    +omments on Issues

    *his is an interesting observation, clearl1 and

    directl1 stated$ Dut what does it have to do

    with the product under review5 It might beworth millions, but it doesn3t belong on the

    Issues 7ist for this product$ It should go on

    the Related issues list$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    48/59

    'ore issues *he price/performance table should be sorted

    using either Luicksort or Shell sort$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    49/59

    +omments

    Raise issues, but avoid all temptation to giveadvice in the Issues 7ist$ *he1 won3t be

    welcome if the1 come in that form, so if1ou reall1 must give advice, find someunofficial vehicle$ *ake the producers tolunch, or out for a beer, before 1ou share

    1our vast e@perience on matters of sorting$If 1our idea isn3t worth the price of a beer,wh1 not forget it5

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    50/59

    'ore issues

    *he three diagrams drawn b1 )arold 'itter

    are not in the standard format uired for such diagrams in our

    installation$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    51/59

    +omments

    *his point is nicel1 specific, but wh1 do we

    have to mention poor )arold5 We are

    reviewing the product, not the people$ Findanother wa1 to identif1 the diagrams and

    leave people3s names out of the Issues 7ist$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    52/59

    'ore issues O

    *he committee was unable to understand the

    significance of paragraph 8$$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    53/59

    +omments O

    4othing3s wrong with this one$ It3s specific aboutwhich paragraph is under discussion, and it sa1sthe committee doesn3t understand it$ Who can

    argue with that5 et, surprisingl1, people seem tobe afraid to e@press issues this wa1 C We don3tunderstand ?B #on3t worr1 about being thoughtstupid$

    If 1ou don3t understand it, it3s at least a potentialissue in documentation$ And besides, lack ofunderstanding ma1 mean there is somethingdreadfull1 wrong$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    54/59

    et more issues -

    *he bubble sort used for sorting the table of

    price/performance figures is a stupid

    approach if the table should grow an1

    bigger than the present 6! entries$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    55/59

    +omments -

    *he word stupidB doesn3t add an1thing at all and

    might antagoniPe the producers$ *ake it out$ *hen

    tr1 to e@press factuall1 and >uantitativel1 wh1 the

    method is inappropriate for larger tables$

    And if the approach isstupid, what of it5 As Arthur

    +$ +larke e@pressed it, It has 1et to be proved

    that intelligence has an1 survival value$B

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    56/59

    et more issues

    If the D 4A'H option is not used, the

    structuring of the structure operands must

    be e>uivalent to the structuring of the

    structures in the arra1s of structures$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    57/59

    +omments

    *his wasn3t reall1 taken from an Issues 7ist,

    but from an old %7/I manual$ Still, I3ve read

    real issues that were almost as obscure$ I3dadvise 1ou to follow the NISS a@iom

    uestion$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    58/59

    et more issues !

    Frieda Sonntag has no computer science

    background, and her e@perience with %7/Iis nil$ She should never have been

    assigned the coding of this module$

  • 8/13/2019 Lecture 16a Walkthroughs.ppt 0

    59/59

    +omments !

    It3s non of the business of the review

    committee who management has assigned

    to particular ;obs$ *he committee3s businessis to review the product and tell what state

    it3s in$ )ow it got to that state is another

    issueTand not for the Issues 7ist$ Who isresponsible is even less of an issue, and

    raising it is sure to be ineffective$


Top Related