1

Project Management and Testing

Phases and Milestones

Budget and Resources

Resource Plan

Resource Plan

Activities and Deliverables

2

Introduction - Goal

Organization

Phases and Milestones

Budget and Resources

Risk

Releases

Project Plan

Project Plan

Activities and Deliverables

Project Plan

System Requirements

System Design

Quality Plan

Resource Plan

Configuration Management

Project Portfolio

Project Portfolio

System Test Plan

…

3

PROPS

managing

operative

supporting

Tollgate (TG0) decision to start prestudy

Requirement engineering

Interviews of customer and/or product management

Analysis

Business case analysis

Customers

Competitors

Costs

Benefits

Risks

Results in an assignment for a feasibility study TG1

Prestudy Phase

4

Tollgate (TG1) decision to start feasibility study

Requirement engineering

User-interface prototyping, specification, use cases, etc

System design

Anathomy, architecture, Implementation Proposals

Simulations using tools or role play

Project planning

Risk analysis

Estimation of size, effort, cost and schedule

Resources, competence, organization, etc

Life-cycle model (prototyping, evolutionary, waterfall, etc)

Results in an assignment for an implementation project TG2

Feasibility Phase

Black BoxRequirements

SystemDesign

SystemDevelopment

SystemTest

SystemValidation

FunctionalSpecification

ActorsUse CasesUse Case DiagramsStoryboardsRequirements Specification

Test PlanTest Cases

Diagrams:- Class- Sequence- Statechart- ActivityDesign PatternsSystem Design

Notes and DetailsSignaturesDesign Patterns

We need to manage the artifacts!

Resource PlanProject Portfolio

Milestones!- Activities- Deliverables- Resources- Responsibles- Dates

5

Builds

Relationship Between Use Cases, Iterations and Builds

Iteration 5

build 5.3

Use case 14

Use case 23

6

Relationship between Use Cases, Iterations and Builds

Iteration 5… 4 … 6

5.45.2 build 5.3

Use case 23

Use case 14

Use case 9 *

Use case 7

*

* «extends» or «includes»

*

Final Code Build and Integration Schedule: a Banking Example

week 31

week 23

weekly code builds biweekly daily

release

baseline

7

week 31

week 23

weekly code builds biweekly daily

releaseModule 1

Module 2

Module 3

Integrationof module

intobaseline

baseline

Plan evolutionary increments of the baseline!

activitytask

Final Code Build and Integration Schedule: a Banking Example

week 31

week 23

weekly code builds biweekly daily

releasebank query module

bank deposit module

bank withdrawal module

Integrationof module

intobaselinetasks

baseline

8

Typical Day-by-Day Code Integration Process

week 25

week 26

week 27

week 28

week 29

week 30

week 31

week 24

week 23

weekly builds biweekly daily

release= overnight regression test

development 6 pm 7 am

Runregression tests

time

development

Freeze development Confirm baseline orrevert to previous baseline

Resource Planning

Month 1

1 2 3 4

Month 2

1 2 3 4

Month 3

1 2 3 4

Month 4

1 2 3 4

Month 5

1 2 3 4

MilestonesRelease to production

Complete testing

Requirements

Design

Freeze requirements

Risk Analysis

2 2 2 3 2 2 4

2 2 2 1 1 1

4 4 4 3 3 4

4 4 4 4 3 3 4 4 4 4 4 3 3 4 4 4 4 4

4 4 4 4 4

4 4

4

Given team size:

To be assigned4

Halvacation

Karenvacation

9

Cyclic, Incremental or Evolutionary Planning

1 2 3 4 5 6 7 8 9 0 1 2 3 4 5

1. strategy2. plan3. requirements4. design5. implementation6. test7. postmortem

MilestonesDelivery

1. strategy…

Iteration 1

Iteration 2

1. strategy…

Cycle 1

Week1

Cycle 2Cycle 3

1 1 1 1 1

Iteration 3

1. The total cost of the project,

e.g., increase expenditures

2. The capabilities of the product,

e.g., subtract from a list of features

3. The quality of the product,

e.g., increase the mean time between failure

4. The date on which the job is completed.

e.g., reduce the schedule by 20%

e.g., postpone project's completion date one month

The Variables of Project Management

10

Projekt

Schedule

Cost

Quality

Functionality

On time, within budget and meet requirements!

The Window of Opportunity

Bullseye Figure for Project Variables

cost

capability duration

defectdensity

Target :$70K

Target : 30 wks

Target : 4 defects/Kloc

Target:100%

11

Bullseye Figure for Project Variables

cost

capability duration

defectdensity

Target :$70K

Actual: 100%

Target : 30 wksTarget :

4 defects/Kloc

thisproject

Actual:1 defect/Kloc

Actual:20 wks

Actual:$90K

Target:100%

Coding versus Documenting

12

Urgent versus Important

Triage in Project Management

• Among top items in importance?– if so, place it in do at once category

• otherwise Ignore without substantially affecting project?– if so, place it in last to do category

• otherwise (Do not spend decision time on this)– place in middle category

13

Delegation

Do it youself!

Delegate Ignore?

Delegate and

follow up!Important

NotImportant

NotUrgentUrgent

Look after your own interests - but be a team player. You are a valuable resource in the company, and sometimes it is important to delegate tasks. However, don't delegate things that you find urgent and important - make sure you take responsibility and show initiative. However, protect yourself from small annoying tasks, unless you are told explicitly to leave other important tasks to focus on it. Even if we might not like it there is an hierarchy at work and you are studying a higher education and will be highly qualified (and well paid) and I'm guessing you will have trainees or other subordinates that are less paid and who can take on things you don't want to do.

14

If you are currently having a full workload (40 hours / week) don't accept additional work without letting the person know that this will delay your other projects. Make sure that your boss is alright with the delay and added cost before accepting a task. There is no shame in admitting that you are already working 100%. Swedish companies today actually try to avoid having people work overtime, at least in the IT sector. The reason is that a person in sick-leave is of no use for the development, it's better to keep developers healthy and happy. That's also why they will ask you about your leisure activities in a job interview - they want to know that you have a way to relieve stress etc. after work so that you can remain healthy at work.

More on Project Management…

• Project in Software Enginnering

• Or, most of the other 4th year projects!

• Dedicated Project Management courses at IES etc. Read them!

15

Assignment 6 – Resource Plan

• The resource plan should include information on which main project phases and activities that are planned (further requirements analysis, software architecture design, testing, etc) for the rest of the project and the amount of resources that are planned for each activity. It should also be expressed when these activies are planned to be active.

• Each main project activity should have a start and stop date (it's ok to say 'month1' or 'day30' instead of a real date, it may actually be preferable).

• Each main project activity should have a planned amount of manhours (1 manyear is 1800 manhours).

• Each main project activity should have a number of staff members allocated to the activity, with one team or activity leader.

• You will propable find that some staff members will need to double in some functions or work with more than one activity. This means that you need to plan the work so that a staff member will not be allocated too much or too less.

• Milestones etc should be visible in the plan.

• Remember that your investor is willing to invest 5 ”manyears” in the development of the company’s first system, so the plan should not exceed that ammount of work (note that 5 manmonths are already spent on your initial project portfolio...). How you plan to use the resources is up to you, but should be described in your resource plan.

• There are licenses of Microsoft Project, which you may use to produce Gantt charts etc if you so prefer. You can also use Microsoft PowerPoint or any general purpose software to do plans graphically.

• Keep It Simple! :-)

Testing...

16

Now

• Testing!– Unit Testing

• Black Box Testing

• White Box Testing

• Test Coverage

– Integration Testing

– System Testing

loss

Development Overview

Requirements

Architecture

System code

loss

loss

loss of information

Interface specsDetailed design

Function code

Module (e.g., package) code

loss

loss

Customer

OK?

17

Golden Rules of Testing

Goal of testing: maximize the number and

severity of defects found per dollar spent

– thus: test early

Limits of testing: Testing can only determine

the presence of defects, never their absence

– use proofs of correctness to establish “absence”

Test Plan

Testing

Sys Req

HL Design

LLDesign

Code

TestCases

TestCases

TestCases

TestCases

1h 800h80h8h

$

18

Artifacts and Roles for Integration Testing

Testengineer

Component engineer

Integration tester

System tester

Use-case model

Test case

Test procedure

Test evaluation

Test plan Test

component

. . . . . . . . . . . . . . . . . . responsible for . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Defect management

Testing: The Big Picture

Methods

Combinations of methods in class

Packages of classes

Use-cases & System Functions

Function

Module

Module combination

2. Integration tests

3. System tests

1.Unittests

19

Elaboration

Unified Process

Inception Construction Transition

Requirements

Analysis

Design

Implemen-tation

Test

Prelim.iterations

Iter.#m

Iter.#m+1

Integration tests

Iter.#1

Iter.#n

Iter.#n+1

…..

Unit tests

Iter.#k

…..

System tests

Black-box Testing

Black box… requirements

Actual outputcompared

withrequired

Input determinedby...

Result

20

White-box Testing

Input determinedby...

Result

White box

…designelements

Confirmationof expected

behavior

Black-, Gray-, & White-box Testing

Black box… requirements

Actual outputcompared

withrequired output

White box

Gray box… requirements &key design elements

Input determined by...Result

…designelements

Confirmationof expected

behavior

As for black-and white box

testing

21

Covering Every Statement is Not Sufficient

u>1 andv==0

x = x/u

u==2 orx>0

++x

No

Yes

Code attempt to implement flowchart

if ( (u>1) && (v==0) ) (1)x = x/u; (2)

if ( (u==2) || (x>3) ) (3)++x; (4)

u=2, v=0 and x=3• executes every line (1) - (4) • gives the correct output x= 2.5

However, line (3) is wrongNo

Yes

Required program

Testing Ranges: Elementary Cases

1. within range

2. at the boundaries

of the range

3. outside the range

(“illegal”)

range

Minimize what you need to testor you will never stop testing!

22

Perform Method Testing 1/2

1. Verify operation at normal parameter values(a black box test based on the unit’s requirements)

2. Verify operation at limit parameter values(black box)

3. Verify operation outside parameter values(black box)

4. Ensure that all instructions execute(statement coverage)

5. Check all paths, including both sides of all branches (decision coverage)

6. Check the use of all called objects7. Verify the handling of all data structures8. Verify the handling of all files

Perform Method Testing 2/2

9. Check normal termination of all loops

(part of a correctness proof)

10. Check abnormal termination of all loops

11. Check normal termination of all recursions

12. Check abnormal termination of all recursions

13. Verify the handling of all error conditions

14. Check timing and synchronization

15. Verify all hardware dependencies

23

• VolumeSubject product to large amounts of input.

• UsabilityMeasure user reaction (e.g., score 1-10).

• PerformanceMeasure speed under various circumstances.

• ConfigurationConfigure to various hardware / software

e.g., measure set-up time.

• Compatibility-- with other designated applications

e.g., measure adaptation time.

• Reliability / AvailabilityMeasure up-time over extended period.

Types of System Tests 1

• SecuritySubject to compromise attempts.

• e.g., measure average time to break in.

• Resource usageMeasure usage of RAM and disk space etc.

• Install-abililtyInstall under various circumstances.

• measure time to install.

• RecoverabilityForce activities that take the application down.

• measure time to recover

• ServiceabililtyService application under various situations.

• measure time to service

• Load / StressSubject to extreme data & event traffic

Types of System Tests 2

24

Key Attributes for Usability Testing

• AccessibilityHow easily can users enter, navigate & exit?

• e.g., measure by average time taken to . . .

• ResponsivenessHow quickly does the application allow the user to

accomplish specified goals?• e.g., measure by average time taken

• EfficiencyDegree to which the number of required steps for

selected functionality is minimal• “minimal” deduced in theory • e.g., measure by minimal time / average time

• ComprehensibilityHow easy is the product to understand and use with

documentation and help?• e.g., measure time taken for standard queries

Alpha- and Beta- Releases

In-house and highly trusted users

– Multiplies testing

– Previews customer reaction

– Benefits third-party developers

– Forestalls competition

Selected customers

– Multiplies testing activity

– Gets customer reaction

Alpha

Beta

25

Roadmap for the Transition Iterations

2. Conduct alpha testing.

1. Plan alpha and beta testing.

• Prepare

• Distribute & install

• Carry out (users / customers)

• Gather defect reports

• Observe stopping criteria

• Correct defects3. Conduct beta testing.

• Define population

• Plan defect collection

• Identify stopping criteria

• Completing a particular test methodology– Complete the procedures of a method or tool.

• Estimated percent coverage for each category– predetermine percent of each & how to calculate– e.g., “95% statement coverage”

• Error detection rate– predetermine rate with given severity level – e.g., “2 medium severity defects or less per 100 hours of

operation”• Total number of errors found

– (if possible) computed from a percentage of remaining defects

– predetermine percent– e.g., “95% of estimated existing defects found ”

Stopping Criteria

26

Target: 95%

Stopping Criteria: Graphical Representation

time

Percentage tested or found

%

Stopping Criteria: Graphical Representation

Error detection rate

Errors foundper 1000 hrs

Target: <= 7 per1000 hrs for 4 weeks

time

27

ANSI/IEEE 829-1983 Software Test Documentation

1. Introduction

2. Test planitems under test, scope, approach, resources, schedule, personnel

3. Test design items to be tested, the approach, the plan in detail

4. Test casessets of inputs and events

5. Test procedures steps for setting up and executing the test cases

6. Test item transmittal reportitems under test, physical location of results, person responsible for transmitting

7. Test logchronological record, physical location of test, tester name

8. Test incident reportdocumentation of any event occurring during testing which requires further investigation

9. Test summary reportsummarizes the above

This is how far you will go in D7025E!

28

Project Plan

System Requirements

System Design

Quality Plan

Resource Plan

Configuration Management

Project Portfolio

Project Portfolio

System Test Plan

…

Future

• Guestlectures– Mikael Börjesson on Project Management and System Design

– Kåre Synnes on Extreme Programming & Code review

– Matthias Kranz on HCI

• Cancelled lecture: Wednesday 3/10

• Code review (workshop 1/10)– Prepare yourself by reading assignment

– (If you use different code, bring materials to the lecture and notify me about the source)

29

Agile workshop

• Agile workshop 2/10 and 8/10– Groups 1-3 on 2/10

– Groups 4-7 on 8/10

– Work will not be done in the standard groups• If someone wants to switch to the other date it is ok but let

me know.

Wednesday

• Quiz #2 on Wednesday. – Covers Design Patterns and this lecture

• Mikael Börjesson is teaching… I hope– Mikael is doing some dental work in the morning

– If Mikael is not teaching then I will talk• Rapid prototyping and Presentation techniques.

30

That’s all folks!

Thanks for your attention!

Range of Errors in Estimating Eventual Cost

Range of cost estimates after conceptualization phase,based on actual cost of $1

$1

Design $1

Implementation $1

Integration/Test $1

Requirementsanalysis $1

Conceptualizationphase Range of cost estimates after

requirements analysis phase25c

$4