Lecture 6: Testing/Quality Lecture 6: Testing/Quality AssuranceAssurance

Damien MarkeyDamien Markey

Lecture 6: Testing/Quality Lecture 6: Testing/Quality AssuranceAssurance

What is Quality Assurance (QA)?What is Quality Assurance (QA)?

QA as a constant process (not just at the QA as a constant process (not just at the end)end)

Testing categoriesTesting categories

Testing planTesting plan

What is Quality What is Quality Assurance (QA)?Assurance (QA)?

Quality Assurance is not testing (although QA Quality Assurance is not testing (although QA does contain testing)does contain testing)

Quality Assurance has many definitions but Quality Assurance has many definitions but generally it is agreed that it isgenerally it is agreed that it is

the application of a process (or processes) to the application of a process (or processes) to assist in creating a quality productassist in creating a quality product

It has been adapted from manufacturing It has been adapted from manufacturing where it has been applied for several where it has been applied for several decadesdecades

See See www.philosophe.comwww.philosophe.com and and www.asq.orgwww.asq.org

What is Quality What is Quality Assurance (QA)-2?Assurance (QA)-2?

The QA lead should be involved from the The QA lead should be involved from the outset of the project so that:outset of the project so that:– They can develop the testing plan (see later) They can develop the testing plan (see later) – They develop an understanding of the client They develop an understanding of the client

and user requirementsand user requirements– They understand the technical requirements They understand the technical requirements

of the technical and hosting teams and the of the technical and hosting teams and the end users. end users.

Common Issues Common Issues With QAWith QA

Commonly QA is either Commonly QA is either – Overlooked and the site is not thoroughly checked Overlooked and the site is not thoroughly checked – not planned properly, causing not planned properly, causing

budget/scope/timeline changesbudget/scope/timeline changes– The time and budget for QA shrinks as “earlier” The time and budget for QA shrinks as “earlier”

tasks slip near delivery datetasks slip near delivery date

Would you watch a TV show that has not been Would you watch a TV show that has not been rehearsed? rehearsed?

Without proper QA you will not deliver a quality Without proper QA you will not deliver a quality sitesite

QA As a Constant QA As a Constant ProcessProcess

QA is not testing at the end of the projectQA is not testing at the end of the project

QA has to be a part of every stage of the QA has to be a part of every stage of the process e.gprocess e.g– HTML coders validate their code before HTML coders validate their code before

passing it to the testing teampassing it to the testing team– Designers test their graphics on PC’s and Designers test their graphics on PC’s and

Macs before handing them to the integratorsMacs before handing them to the integrators

QA as a Constant QA as a Constant Process –2 Process –2

This is called “White Box” testingThis is called “White Box” testing– The programmer knows what to (and what not The programmer knows what to (and what not

to) input into the system and what output to) input into the system and what output should be createdshould be created

If the programmers, designers, coders, If the programmers, designers, coders, integrators and everyone else perform integrators and everyone else perform their white box testing then the QA team their white box testing then the QA team can do their “Black box” testingcan do their “Black box” testing

Black Box TestingBlack Box Testing

The QA lead will produce a test plan based on The QA lead will produce a test plan based on the initial specificationthe initial specification

They can then take the key user paths and key They can then take the key user paths and key tasks that have been documented and test tasks that have been documented and test that those functions perform as designedthat those functions perform as designed

The testers do not need to know how the code The testers do not need to know how the code works as they act as “preliminary users”works as they act as “preliminary users”

Do you need to know how Amazon.com works Do you need to know how Amazon.com works or do you want to order a book?or do you want to order a book?

Testing Testing CategoriesCategories

There are various types of testing that There are various types of testing that should be preformed, depending on the should be preformed, depending on the site, to ensure a quality productsite, to ensure a quality product

These fall into several categories:These fall into several categories:– Unit testsUnit tests– Alpha testingAlpha testing– Content checkContent check– Beta testingBeta testing

Testing Testing Categories (cont.)Categories (cont.)

Load testingLoad testing

Stress testsStress tests

Functional testingFunctional testing

User Acceptance TestingUser Acceptance Testing

Regression testingRegression testing

Security testingSecurity testing

Boundary testingBoundary testing

Unit testingUnit testing

Testing of individual components such as:Testing of individual components such as:– Flash movies, Flash movies, – JavaScript rollovers and JavaScript rollovers and – functions on a pagefunctions on a page

Usually performed by the developers as part Usually performed by the developers as part of their own internal testingof their own internal testing

Performed before the passing of pages to Performed before the passing of pages to the integration teamthe integration team

Alpha TestingAlpha Testing

Testing performed, based on the original Testing performed, based on the original specification, before the site goes livespecification, before the site goes live

Usually performed on the development siteUsually performed on the development site

Usually performed by the internal Usually performed by the internal development teamdevelopment team

Content CheckContent Check

Checks the content (images and text) for the Checks the content (images and text) for the following:following:– Accuracy Accuracy – Placement within site and pagePlacement within site and page– Copy text readabilityCopy text readability– SyntaxSyntax– ImagesImages

Beta TestingBeta Testing

Post integration (and initial deployment) Post integration (and initial deployment) testing on the liver servertesting on the liver server– The site is not “live” at this point but is on the The site is not “live” at this point but is on the

live server in the live server set-uplive server in the live server set-up

Final testing of the site against the original Final testing of the site against the original specificationspecification

Load TestingLoad Testing

What would happen if your site suddenly What would happen if your site suddenly becomes very popular? E.g.becomes very popular? E.g.– Friendsreunited.co.uk went from 5,000 page Friendsreunited.co.uk went from 5,000 page

impressions a month to 50,000 a month in impressions a month to 50,000 a month in about three monthsabout three months

Does the site say “server down” or does it Does the site say “server down” or does it handle the increased load gracefully by handle the increased load gracefully by placing a “Call back later” sign upplacing a “Call back later” sign up

Or do files become corrupted and data lost?Or do files become corrupted and data lost?

Load Testing Load Testing ToolsTools

There are software tools for load testingThere are software tools for load testing

These simulate the effect of a large number of These simulate the effect of a large number of users on a siteusers on a site

They can also simulate a lot of people They can also simulate a lot of people performing certain tasksperforming certain tasks

Tools such as webload and Mercury Tools such as webload and Mercury Interactive perform such testingInteractive perform such testing

Needs to be carried out for large load site e.g. Needs to be carried out for large load site e.g. Big Brother and comic relief sitesBig Brother and comic relief sites

Stress TestsStress Tests

These are similar to load test but on the These are similar to load test but on the browser end of the chainbrowser end of the chain

They work by seeing the response of the They work by seeing the response of the site on a low spec, low bandwidth site on a low spec, low bandwidth connection with older browsersconnection with older browsers– I.e. how does your site work on a Pentium I.e. how does your site work on a Pentium

200Mhz with 8MB of RAM using IE 3?200Mhz with 8MB of RAM using IE 3?

Functional Functional TestingTesting

These are tests that do not involve looking at code These are tests that do not involve looking at code but concentrate on the specified functionality of but concentrate on the specified functionality of the sitethe site

Testers act as users and try to perform Testers act as users and try to perform representative tasksrepresentative tasks

As all tasks and permutations cannot be carried out, As all tasks and permutations cannot be carried out, representative tasks group several tasks together representative tasks group several tasks together e.g.e.g.– User selects an item, places it in the baskets and then User selects an item, places it in the baskets and then

goes back to the shop. They then return to the shop goes back to the shop. They then return to the shop later to checkoutlater to checkout

User Acceptance User Acceptance TestingTesting

Tests are based on the client specificationTests are based on the client specification

Usually test the Usually test the – Platforms Platforms – BrowsersBrowsers– Operating SystemOperating System– Functionality of siteFunctionality of site

Once passed the client can “sign off” the siteOnce passed the client can “sign off” the site

Regression Regression TestingTesting

States what will be tested on the second, third and States what will be tested on the second, third and any further iterations of testing I.e.any further iterations of testing I.e.– Core functionality is tested on the first, second and final Core functionality is tested on the first, second and final

passpass– Flash content is checked in first and secondary testing Flash content is checked in first and secondary testing

(if bugs fixed by second test)(if bugs fixed by second test)

This is to prevent the possibility of fixes in bugs for This is to prevent the possibility of fixes in bugs for one section causing new bugs in previously fixed one section causing new bugs in previously fixed codecode

This is essential as most modern sites have shared This is essential as most modern sites have shared code between sectionscode between sections

Security TestingSecurity Testing

This testing check that data sent along This testing check that data sent along secure links is safe from unauthorised secure links is safe from unauthorised accessaccess

Involves attempting to access the secure Involves attempting to access the secure data in transit and stored on the servers data in transit and stored on the servers and databasesand databases

Essential for financial transaction sitesEssential for financial transaction sites

Boundary TestingBoundary Testing

Tests forms and input methods for data that is Tests forms and input methods for data that is not specified in the original specification e.g.not specified in the original specification e.g.– Entering dates into an amount fieldEntering dates into an amount field– Entering numbers into a name fieldEntering numbers into a name field– Entering “guess” page names into the URL of Entering “guess” page names into the URL of

the sitethe site

You do not want a site that takes money to You do not want a site that takes money to not throw back a response if a name is not throw back a response if a name is entered in the credit card field!entered in the credit card field!

Testing PlanTesting Plan

All of the categories and types of test are put into All of the categories and types of test are put into the project test planthe project test plan

This details the followingThis details the following– Scope of testingScope of testing

Which type of test will be used, what platforms will be tested Which type of test will be used, what platforms will be tested and what environment will be usedand what environment will be used

– Bug categoriesBug categoriesScales (e.g. 1 to 3 with 1 being severe) and types (e.g Scales (e.g. 1 to 3 with 1 being severe) and types (e.g graphical, functional, usability etc)graphical, functional, usability etc)

Testing Plan - 2Testing Plan - 2

Sections contSections cont– Configuration ManagementConfiguration Management

The file naming and versions to be usedThe file naming and versions to be used

– Risks and AssumptionsRisks and AssumptionsRisks such as what can happen if major errors are Risks such as what can happen if major errors are found found

What resources and pre-requisites are needed for What resources and pre-requisites are needed for testingtesting

– Schedule and no of iterationsSchedule and no of iterations

Testing Plan - 3Testing Plan - 3

Test resourcesTest resources– What internal testing resources will be neededWhat internal testing resources will be needed

Testing casesTesting cases– The cases to be used (usually key paths)The cases to be used (usually key paths)– Any extra Representative casesAny extra Representative cases

Final deliverablesFinal deliverables– Bug report – a list of bugs, severity, type and Bug report – a list of bugs, severity, type and

descriptiondescription

QA summaryQA summary

A summary of bugs found at end of A summary of bugs found at end of developmentdevelopment

Any outstanding issuesAny outstanding issues

Any recommendations form the testing teamAny recommendations form the testing team