lecture 16a walkthroughs

of 59 /59
Software Reviews, Walkthroughs, Inspections, and Audits Better to find an error twice than never to find it at all. [Freedman/Weinberg 90] Freedman, D.P., G.M. Weinberg, "Handbook of Update terminology according Give references to O’Neil 

Author: govind-kamboj

Post on 03-Jun-2018

221 views

Category:

Documents


0 download

Embed Size (px)

TRANSCRIPT

  • 8/12/2019 Lecture 16a Walkthroughs

    1/59

    Software Reviews,

    Walkthroughs, Inspections, and

    AuditsBetter to find an error twice than

    never to find it at all.

    [Freedman/Weinberg 90] Freedman,

    D.P., G.M. Weinberg, "Handbook of

    Update terminology

    according

    Give references

    to ONeil

  • 8/12/2019 Lecture 16a Walkthroughs

    2/59

    Why Do We Have Formal

    Technical Reviews?

    1. To err is human.2. Lots of errors escape the originator more

    easily than anyone else.

    3. Reviews are educational.

  • 8/12/2019 Lecture 16a Walkthroughs

    3/59

    Purpose: Provide

    visibility into state of project

    opportunities for personnel (project and

    non-project) to discuss topics related to

    project

    intermediate milestones and sense of

    progress

    assessment of technical adequacy of project

  • 8/12/2019 Lecture 16a Walkthroughs

    4/59

    Use the power of a team to

    Point out needed improvements of a

    product.

    Confirm the parts in which improvement is

    not needed or desired.

    Make technical work more uniform and

    predictable in quality, which makes it moremanageable.

  • 8/12/2019 Lecture 16a Walkthroughs

    5/59

    Technical review is different

    from other reviews For example, budget and management

    reviews

    Technical review answers question: will this

    product do the job its supposed to do

    If answer is no, then

    no schedule is on time, and

    no cost is cheap enough.

  • 8/12/2019 Lecture 16a Walkthroughs

    6/59

    When?

    Single review that occurs on a particular date

    Single review that occurs in response to a

    particular condition Multiple reviews that occur periodically: e.g.,

    monthly

    Multiple reviews that occur in response to aparticular condition, e.g., code review forsubsystem

  • 8/12/2019 Lecture 16a Walkthroughs

    7/59

    Benefits

    10X reduction in errors in products

    50-80% cost reduction

  • 8/12/2019 Lecture 16a Walkthroughs

    8/59

  • 8/12/2019 Lecture 16a Walkthroughs

    9/59

    Why not just pass it around and

    have reviewers sign off

    Comments are too large Comments are unread

    Reviewers dont have to take responsibility

    for the content.

  • 8/12/2019 Lecture 16a Walkthroughs

    10/59

    Requirements

    Team participation

    Documented procedures

    Documented results

  • 8/12/2019 Lecture 16a Walkthroughs

    11/59

    Types of Reviews

    Management Review

    Technical Review

    Walkthrough

    Inspection

    Audit

  • 8/12/2019 Lecture 16a Walkthroughs

    12/59

  • 8/12/2019 Lecture 16a Walkthroughs

    13/59

  • 8/12/2019 Lecture 16a Walkthroughs

    14/59

    Types of Reviews

    Management Review

    Technical Review

    Focus on specification and design

    Determine whether products meet specs

    Walkthrough

    Inspection

    Audit

  • 8/12/2019 Lecture 16a Walkthroughs

    15/59

    Scenario for a typical review

    Leader: Welcome. We are here to review X for the

    purpose of Y.

    Each person is introduced. The recorder is introduced.

    An agenda is presented and the review process

    explained.

    The list of material in the review packet is stated.

    The review begins

  • 8/12/2019 Lecture 16a Walkthroughs

    16/59

    Review

    Agenda set by leader, agreed to by

    committee

    Review proceeds line-by-line.

    Reviewers bring up issues as they encounter

    them.

    Issues are added to an Issues List.

    Agenda changes as issues are encountered.

  • 8/12/2019 Lecture 16a Walkthroughs

    17/59

    Round Robin Review

    Force every member to participate.

    Each person takes the lead for a section:

    paragraph,

    line of code,

    function,

    test case

    Good way to get involvement from team.

  • 8/12/2019 Lecture 16a Walkthroughs

    18/59

    Types of Reviews

    Management Review

    Technical Review

    Walkthrough Used to find anomalies and evaluate conformance

    Typically used for source code

    Effective form of education

    Inspection

    Audit

  • 8/12/2019 Lecture 16a Walkthroughs

    19/59

    Walkthroughs

    Producerguides review

    Step by step presentation of product:

    Code, design, report, test cases Can cover lots of material

    Can have more people attend, less prepared

    More work for presenter

    May be difficult to control interactions

    Too many interruptions

    Always hard to have a producer present

  • 8/12/2019 Lecture 16a Walkthroughs

    20/59

    Walkthroughs

    Participant should spend 1 hour preparing

    for each hour in meeting

    Should be scheduled to be brief

    Should only review completed code or

    document

    Should require participants to sign report

  • 8/12/2019 Lecture 16a Walkthroughs

    21/59

    Types of Reviews

    Management Review

    Technical Review

    Walkthrough

    Inspection

    More formal type of walkthrough

    Used to identify specific errors or problems

    Audit

  • 8/12/2019 Lecture 16a Walkthroughs

    22/59

  • 8/12/2019 Lecture 16a Walkthroughs

    23/59

    Types of Reviews

    Management Review

    Technical Review

    Walkthrough

    Inspection

    Audit

    Independent evaluation of process or product

    Ensures that the process is being followed

  • 8/12/2019 Lecture 16a Walkthroughs

    24/59

    Review Teams

    Permanent or semi-permanent teams

    IV&V or QA teams

    Get to be very good at reviews

    Need to have some way of reviewing the

    reviewers.

  • 8/12/2019 Lecture 16a Walkthroughs

    25/59

    The team Leader

    Job: obtain a good review or report why it wasnot possible

    Reflects the quality of the review process,not the quality of the product

    Must be accurate

    Recorder

    Provide info for accurate report of review

    Short, public notesCapture essence of each issue

    Must ensure group has reached conclusion

    Dont video tape

  • 8/12/2019 Lecture 16a Walkthroughs

    26/59

    Tasks of leader

    Monitor preparedness of team members

    Set pace

    Keep meeting on track Poll members to reach consensus

    Ensure participation

    Has right to terminate review if it is unproductiveDisagreements

    Fatal flaws

  • 8/12/2019 Lecture 16a Walkthroughs

    27/59

    Consensus

    Decision of the group is equal to the most

    severe opinion of the group members.

    Be conservative.

    Accept the doubts of the most doubting

    member.

    No I told you so, but I got outvoted.

  • 8/12/2019 Lecture 16a Walkthroughs

    28/59

    Reviewers

    Must be qualified to contribute

    Must be unbiased

    Dont invite management if it causes

    conflict

    The point is to review the project, not the

    producers.

  • 8/12/2019 Lecture 16a Walkthroughs

    29/59

    Rules for Reviewers

    I. Be prepared

    II. Evaluate product, not people.

    III. Watch your language.

    IV. One positive, one negative.

    V. Raise issues, dont resolve them.

    VI. Record all issues in public.

    VII. Stick to technical issues.

  • 8/12/2019 Lecture 16a Walkthroughs

    30/59

    Rules for Reviewers

    If you find something, assume its a mistake.These are your peers, not your enemy. Remember

    people are involved

    Avoid why did you why didnt you say

    instead I dont understand Why did you set the upper bound to 10 here?

    I dont understand why the upper bound is 10 here.

    seems trivial, but its not.

    Do not get bogged down in style issues. Forexample, if efficiency isnt an issue, then dontmake it one.

    If it makes things less maintainable, thats an issue.

    If there are standards, then either stick to thestandard, or dispose of the standard.

  • 8/12/2019 Lecture 16a Walkthroughs

    31/59

  • 8/12/2019 Lecture 16a Walkthroughs

    32/59

    Time

    At most 2 hours at a time

    Come back again if necessary

    Depends on

    Complexity

    Size of project

    Closeness to completion

  • 8/12/2019 Lecture 16a Walkthroughs

    33/59

    Everyone must prepare!

    Review packet

    Everything relevant to making correct

    judgment: code, specs, test data, results,standards, designs,

    80% of failures stem from unprepared teams

  • 8/12/2019 Lecture 16a Walkthroughs

    34/59

    Tactics for participation

    Devils Advocate

    One person makes strongest case against aproduct

    Job is to find fault

    Needs to be an actor Debugging

    Producers leave known bugs in

    Quality of review depends on how many found

    Alarms

    Time each persons contribution, cut off

    Stand-up Reviews

    Talk as long as you stand on one leg

  • 8/12/2019 Lecture 16a Walkthroughs

    35/59

    Report

    Goal: Does the product do the job its supposed todo?

    Complete.

    Correct.

    Dependable as basis for other work

    Measurable for purposes of tracking.

    What was reviewed? Signatures of reviewers.

    Leader and recorder.

  • 8/12/2019 Lecture 16a Walkthroughs

    36/59

    Notes

    Requires learning: to review and to write

    Need to allocate time

    Not a form of project management

    But can provide information for tracking

    Review early and often

    But not too often

  • 8/12/2019 Lecture 16a Walkthroughs

    37/59

    Need for Reviews

    Reviews help organization deliver qualityproduct

    Examine individual parts early in processIdentify problems spanning multiple items or

    phases

    Reviews help identify problems early

    Reviews help build image of the productand the vision of the product in the minds ofthe team members

  • 8/12/2019 Lecture 16a Walkthroughs

    38/59

    Cost of Software Reviews

    Up-front cost for Training

    Staff preparation

    Review Conduct

    Code review may be 8-15% of total cost of project

    Offset by savings: reduced rework later in project(25-35% of total project cost saved) Raytheon: pre-1988: 43% of software cost in defect

    correction 1994 (after software reviews instituted): 5% of project

    cost in defect correction

  • 8/12/2019 Lecture 16a Walkthroughs

    39/59

    Writing Issues

    Groups of 3

    Ill give you an issue from an issue list,

    You tell me whats wrong with it.

    Issues (and responses) fromHandbook of

    Walkthroughs, Inspections, and Technical

    Reviews, Daniel Freedman and Gerald

    Weinberg, 3rdEd, Dorset House.

  • 8/12/2019 Lecture 16a Walkthroughs

    40/59

    Writing Issues #1

    Some of the explanations of user commands

    were misinterpreted by members of the

    review committee.

  • 8/12/2019 Lecture 16a Walkthroughs

    41/59

    Comments on Issues #1

    Not specific enough. Whichexplanations were

    misinterpreted? Dont make the producers

    guess, or they may change the wrong thing.

  • 8/12/2019 Lecture 16a Walkthroughs

    42/59

  • 8/12/2019 Lecture 16a Walkthroughs

    43/59

    Comments on Issues #2

    In writing, clarity isnt the most important

    thing: its the onlything. If you dont want

    to be not understood, dont never use nodouble negatives.

    Another thingwhat is the issue anyway? Is

    the maximum too low? Too high? Or is itthat some of the codes should have been

    made not eligible? Or eligible? Be direct.

  • 8/12/2019 Lecture 16a Walkthroughs

    44/59

    Writing Issues #3

    The referenced table of constant values was

    not part of the review packet.

  • 8/12/2019 Lecture 16a Walkthroughs

    45/59

    Comments on Issues #3

    Always give the most direct reference

    available. There could be more than one

    table, now or in the future. Why make theproducers search?

  • 8/12/2019 Lecture 16a Walkthroughs

    46/59

    Writing Issues #4

    The method used for maintaining the message

    queue seems to solve a severe performanceproblem weve been having with the

    production version of the RKY system.

  • 8/12/2019 Lecture 16a Walkthroughs

    47/59

    Comments on Issues #4

    This is an interesting observation, clearly and

    directly stated. But what does it have to do

    with the product under review? It might beworth millions, but it doesnt belong on the

    Issues List for this product. It should go on

    the Related issues list.

  • 8/12/2019 Lecture 16a Walkthroughs

    48/59

    More issues #5The price/performance table should be sorted

    using either Quicksort or Shell sort.

  • 8/12/2019 Lecture 16a Walkthroughs

    49/59

    Comments #5

    Raise issues, but avoid all temptation to giveadvice in the Issues List. They wont be

    welcome if they come in that form, so ifyou really must give advice, find someunofficial vehicle. Take the producers tolunch, or out for a beer, before you share

    your vast experience on matters of sorting.If your idea isnt worth the price of a beer,why not forget it?

  • 8/12/2019 Lecture 16a Walkthroughs

    50/59

    More issues #6

    The three diagrams drawn by Harold Mitter

    are not in the standard format (DS-109)

    required for such diagrams in our

    installation.

  • 8/12/2019 Lecture 16a Walkthroughs

    51/59

    Comments #6

    This point is nicely specific, but why do we

    have to mention poor Harold? We are

    reviewing the product, not the people. Findanother way to identify the diagrams and

    leave peoples names out of the Issues List.

  • 8/12/2019 Lecture 16a Walkthroughs

    52/59

    More issues #7

    The committee was unable to understand the

    significance of paragraph 3.1.

  • 8/12/2019 Lecture 16a Walkthroughs

    53/59

    Comments #7

    Nothings wrong with this one. Its specific aboutwhich paragraph is under discussion, and it saysthe committee doesnt understand it. Who can

    argue with that? Yet, surprisingly, people seem tobe afraid to express issues this wayWe dontunderstand Dont worry about being thoughtstupid.

    If you dont understand it, its at least a potentialissue in documentation. And besides, lack ofunderstanding may mean there is somethingdreadfully wrong.

  • 8/12/2019 Lecture 16a Walkthroughs

    54/59

    Yet more issues #8

    The bubble sort used for sorting the table of

    price/performance figures is a stupid

    approach if the table should grow any

    bigger than the present 20 entries.

  • 8/12/2019 Lecture 16a Walkthroughs

    55/59

    Comments #8

    The word stupid doesnt add anything at all and

    might antagonize the producers. Take it out. Then

    try to express factually and quantitatively why the

    method is inappropriate for larger tables.

    And if the approach isstupid, what of it? As Arthur

    C. Clarke expressed it, It has yet to be proved

    that intelligence has any survival value.

  • 8/12/2019 Lecture 16a Walkthroughs

    56/59

    Yet more issues #9

    If the BY NAME option is not used, the

    structuring of the structure operands must

    be equivalent to the structuring of the

    structures in the arrays of structures.

  • 8/12/2019 Lecture 16a Walkthroughs

    57/59

    Comments #9

    This wasnt really taken from an Issues List,

    but from an old PL/I manual. Still, Ive read

    real issues that were almost as obscure. Idadvise you to follow the KISS axiom

    (Keep It Simple, Stupid), except that I

    would be contradicting my own advice fromthe previous question.

  • 8/12/2019 Lecture 16a Walkthroughs

    58/59

    Yet more issues #10

    Frieda Sonntag has no computer science

    background, and her experience with PL/Iis nil. She should never have been

    assigned the coding of this module.

  • 8/12/2019 Lecture 16a Walkthroughs

    59/59

    Comments #10

    Its non of the business of the review

    committee who management has assigned

    to particular jobs. The committees businessis to review the product and tell what state

    its in. How it got to that state is another

    issueand not for the Issues List. Who isresponsible is even less of an issue, and

    raising it is sure to be ineffective.