qccommon rq

7/24/2019 QCCommon RQ

1/18

Last Updated Date: 2/18/08

Author: Ravi Sharda

QCCommon & RQ Issues

1. INTRODUCTION..................................................................................................................................2

1.1. DOCUMENTOVERVIEW.....................................................................................................................2

1.2. RQ & COMMONOVERVIEW..............................................................................................................2

1.2.1. RQ (Requirements) System Overview......................................................................................2

1.2.2. QCCommon Overview.............................................................................................................4

1.2.3. QOA and QCCommon System Context (Networx)...................................................................1.2.4. QOA and QCCommon !e"#oyment !ia$ram..........................................................................%

2. QCCOMMON ISSUES..........................................................................................................................7

2.1. USERINTERFACE& USABILITY........................................................................................................7

2.1.1. &i$'t Cou"#in$ o * +ode in ,ava C#asses..............................................................................%

2.1.2. &oo -u+' o * o$i+ is in ,ava S+ri"ts................................................................................../

2.2. PERFORMANCE

..................................................................................................................................82.2.1. *ssues in QCCommon A""#i+ation +ausin$ #ow "erorman+e................................................../

2.2.2. 0nvironmenta# erorman+e &unin$........................................................................................2.3. GENERALCODECHARACTERISTICS ................................................................................................10

2.3.1. S"a$'etti +ode and si$nii+ant #os in t'e +ode...................................................................12.3.2. &oo many data "oints ex+'an$ed etween QOA and QCCommon via 5&& Request..........12

2.. TESTABILITY...................................................................................................................................13

2.4.1. a+6 o Automated &estin$.......................................................Error! Bookmark not defined.

3. DEPLOYMENT ENVIRONMENT....................................................................................................14

3.1. DEPLOYMENTISSUES......................................................................................................................1

3.2. RQ GRAPHGENERATIONRELATEDISSUESUNCOVEREDTOOLATE.................................................1!

4. PRODUCT DEFINITION & RQ DATA............................................................................................16

.1. UNITTESTINGOFRQ DEFINITION..................................................................................................1"

.2. PRICINGERRORSCAUGHTATTESTINGTIME...................................................................................1"

4.2.1. oor &oo#in$ or 7iewin$8 0ditin$ 9 nit &estin$ o RQ......................................................1%.3. OTHER#NOWNISSUES...................................................................................................................17

4.3.1. !e"enden+y o rovisionin$ :or6#ow (COR0;e


2/18

1. Introduction

1.1. Document Overview

1.2. RQ & Common Overview

RQ tables store Product definitions (also known as Product Builds orProductCatalog). Product definitions govern what is presented at the inception of the order

how a !uote is priced and billed and how downstrea" fulfill"ent activities aree#ecuted.

QCCo""on i"ple"ents the $Product Configurator% concept. & QCCo""on batchapplication retrieves and stores a local cop' of product definition data and

configuration ele"ents for efficient searches. &t runti"e QCCo""on paints theproduct configuration data points and validates user entries upfront ("ostl' as

entered).

Both RQ and QCCo""on have a generic and fle#ible design b' wa' of which new

products or updates to e#isting products can be supported with "ini"u"

custo"iations. *pfront validations and product definition and billing data integrit'reduce errors.

+n the flipside generic architecture adds to the co"ple#it' of the $Product Catalog

, Configurator% design. -aor: testing the co""ent

/esign is design

1.2.1. RQ (Requirements) System Overview

Based on inputs fro" Product -anagers and Billing &nal'sts billing0sales productdefinition is entered in RQ using the Product Builder application in ter"s of

Product (Custo"er Brand)

ervice Categories (Custo"er Product)

Co"ponent 2roups (Custo"er Product Co"ponent or 3eature 2roups)

Co"ponents (3eatures)

values and attribute

values and attribure critical: values and attribute

Based on inputs fro" Provisioning specialists provisioning data definitions is enteredin RQ using the Product Builder &pplication.

Rate tables and algorith"s are also specified using Product Builder application.

&n RQ tree looks as follows as seen using the QCCo""on RQ 4iewer. & sa"ple

graph portion for a 5elephone 6ine service categor' is given below.

Page:7


3/18

Figure 1 Sample Product Dei!itio! "as vie#ed via $%%ommo! &ie#er'

Configuration rules are defined in RQ using $/ecisions% such as B89 BC9 and//6B in the above graph. 5here are "ore than ; such decisions in RQ.

Configuration rules could be things like:

& co"ponent is "andator' for a co"ponent group

-andator' attributes of a Co"ponent 2roup or Co"ponent

Restrictions on selectable0enterable values of an &ttribute (possibl' based on

other &ttributes< values

8on6&5= and =ndividual Case Basis (=CB) cases of pricingwhat is used

=dentification of Contract Constrained

>hre are the constraints

&gain what are the constraints-aor: this is good for nothing co""ent

o"e of the decisions are:

98&B (9nable) 98&B decision helps to enable or disable an entit' based on

a condition (defines as attribute path value e#pression in RQ).

65C (6i"it 5otal Children) 5his decision controls the nu"ber of instances of

the child entit' allowed at runti"e. 3or e#a"ple the /9/?? custo"er

Page:@


4/18

product (service categor') is li"ited to one instance under an instance of acusto"er brand (product account) b' placing a 65C decision between the

brand and custo"er product nodes in the product build.

ee above for 65C

//6B (/ropdown 6ist bo# *ser =nterface) A 5his is a *= decision. 5his decision

presents a dropdown list bo# control to accept the selection of a value for its

parent node. 8ote that although this decision (like other *= decisions)specifies a *= control it doesn


5/18

Figure 2 Sample (!d Poi!t %o!iguratio! Scree!

+nscreen events are recognied b' />R and control is passed to the />R ava

class in the backend.

9#plain /ecorators.

Fou "ight want to e#plain different scenarios in which the 9PC page is loaded like

the +& "ode +9C -ode and also the -&C/ scenario where the 9PC page has a 7

i"age configuration.

Page:G


6/18

1.2.3. QOA nd QCCommon System Conte!t ("etwor!)

Figure ) $*A + $%%ommo! S,stems %o!te-t

Page:H


7/18

1.2.#. QOA nd QCCommon De$%oyment Dirm

Figure . Deplo,me!t Diagram

2. QCCommon Issues

2.1. 'ser Interce & 'si%ity

2.1.1. *i+t Cou$%in o 'I code in ,v C%sses

%o!te-t:

5he />R &a# i"ple"entation in QCCo""on is used to recognie onscreen events

and pass the control to the server side as'nchronousl' with infor"ation about the

change in *= controls or order configuration sub"ission. 5he response fro" theserver side contains the D5-6 code related to data ele"ents to be updated.

Configuration rules for a product are defined via $RQ /ecisions% such as B89 //6B

98&3 P&4 etc. 5he />R &&? handler ava class invokes a factor' i"ple"entationthat helps traverse down the graph. $/ecorator% classes are recursivel' invoked for

the current node (in the graph) in !uestion as well as child nodes. 3or *= decisionssuch as //6B D5-6 portions are generated and passed back to the &&? avacript

which in turn updates the *=.

Page:I


8/18

Prolem:

& large a"ount of presentation la'er $displa'% code is actuall' e"bedded in thebackend do"ain la'er distributed a"ong the a"ong the various $/o"ain 6a'er%

ava classes. -ore specificall' do"ain code (for si"plicit' all code other than

presentation and data access) "akes references to the *= code.

o"e of the proble"s with this:

4er' difficult and ti"e consu"ing to change presentation look and feel.

8ot possible to support "ultiple presentations0applications on the sa"e

do"ain code.

3or e#a"ple in the federal space supporting an ?-6 based order entr' "a'beco"e a re!uire"ent or to support >&P and R=- D5-6 for" factors. &

"ore realistic e#a"ple is: 9vents -anager application or Provisioning>orkflow application re!uiring order configuration for a re!uest in order to

show in the +rder tatus *=.

ignificantl' reduced testabilit'. 9ver'thing is tested using browserbased

presentations. &uto"ation tools that capture "ouse clicks and D55P re!uestsare usuall' not ver' useful in a d'na"ic environ"ent because the resulting

"acros are trick' to "aintain. /riving auto"ated tests through direct functioncalls is far easier.

Solutio!:

2.1.2. *oo -uc+ o 'I oic is in ,v Scri$ts

%o!te-t:

&&? (using />R 3ra"ework) is being used for *= rendering and validation ofbilling0pricing and provisioning i"pacting data points. &&? inherentl' uses

avacript calls fro" client to server.

avacript used in QCCo""on

2.2. /erormnce

2.2.1. Issues in QCCommon A$$%iction cusin %ow $erormnce%o!te-t:

&lso there hasn


9/18

usuall' takes seconds to show up on the *=. 5he real culprit is the $5arriffsDelper%service which takes I seconds to respond.

Prolem:

6onglived and large obect graphs in the session. 5he' are not cleared auto"aticall'

when the user "oves out to other applications.

o"e of the services invoked b' QCCo""on have unacceptable high response

ti"es.

Redundant processing (partl' attributable to shortcuts and generic graph processing

structure) see"s to be adding to response ti"es.

Solutio!:

a) /evise a "echanis" to reduce data stored in the session. Re"ove sessiondata when user goes out of the application especiall' since large obect

graphs are currentl' stored in session.

b) Profile the application using a co""erciall' available tool (for e#a"ple

Probe or +pti"ie=t) to pinpoint bottlenecks in the application that result intoeither high CP* utiliation or high contention for shared resources.

c) =dentif' and solve the root cause of the redundant processing.

d) *tilie 6R* caching where appropriate for caching service outputs

2.2.2. 0nvironment% /erormnce *unin

%o!te-t:

&s shown in 3ig E the user hits Q+&0QCCo""on application as follows:

*ser J &pache D55P erver (in the /-K) J Portal erver Cluster JQuoting0+rdering &pache D55P erver J Quoting0+rdering &pplication erver

=t has been observed and proved in 8etwor# that if the user hits Q+&0QCCo""on

directl' the perfor"ance is "uch better than when hitting the sa"e through theportal. &lso li"ited environ"ent tuning has been done since the focus has been

"ore to provide re!uired functionalit'.

Prolem:

-ost of the Q+& and Portal environ"ents are not welltuned 'et. & stagingenviron"ent where test operations tea"s along with developers and testers can

work to tune the s'ste" is usuall' not available.

5here has been a lack of progra"side perfor"ance re!uire"ents in 8etwor# and/P due to which "ost perfor"ance enhance"ents initiatives have been undertakenin silos reducing their effectiveness. 3or e#a"ple +rder 9ntr' s'ste"s such as Q+&

and QCCo""on use a nu"ber of services fro" +9 6=- Portal 5>=? C=PCC =P etc. *nless those s'ste"s are tuned for perfor"ance too the overall

average response ti"es in Q+&0QCCo""on screens would not go down significantl'.

Solutio!:

Page:L


10/18

Perfor" environ"ent tuning for the following in Q.Control Portal and Q+&:

&pache D55P ervers >eblgic ervers 4-s and network links. -onitor

"e"or' usage CP* utiliation and =0+ usage under high load and perfor"environ"ental opti"iations. Currentl' the s'ste"s are not tunes.

Dave perfor"ance re!uire"ents for "ost applications in the progra". 6&

"etrics for services should be established and "easured.

5une co"ple# stored procedures and !ueries.

2.3. ener% Code C+rcteristics

2.3.1. S$+etti code nd siniicnt %os in t+e code

%o!te-t:

&t the core of QCCo""on the code has been designed for generic recursiveprocessing of RQ decisions and the resulting configurator and validations logic. 5o

enable recursive and generic processing design patterns such as /ecorator Patternand 4isitor Pattern have been applied. &lthough these patterns provide an elegant

design the' also bring in so"e a"ount of co"ple#ities in the for" of:

/ecorator Pattern A *suall' brings in too "an' obects that look alike but

work differentl'

4isitor Pattern A Class hierarchies beco"e un"anageable and classes

co"pro"ise encapsulation.

-oreover co"ple#it' also arises fro" the fact that RQ decision tree which is a

co"ple# tree is processes genericall' b' the QCCo""on core. 5herefore ust b'

looking at the code it


11/18

inadvertentl' cause other issues not directl' co"prehensible looking at the code.ince QCCo""on is a fra"ework for a generic configurator the i""ediate issue

"a' get fi#ed but "ight cause issues for a specific co"bination for a differentproduct etc.

Prolem:

o"e classes own too "uch of responsibilit' and are too co"ple#. 9arlier developersused to regularl' sa' that the code is un"aintainable and it is ver' difficult to solveissues without NN. 5he theor' was confir"ed with the "etrics of the code shown

above too.

Root %ause A!al,sis:

Prolem Possile %ou!termeasures

paghetti code and proceduralprogra""ing using obects.

ignificant refactoring of code is re!uired:

5o reduce code co"ple#it' of certain

classes to si"pler and reusableclasses and "ethods

Put functionalit' where it trul' belongs

h, 9#perience level of developers was

"ostl' low

3ocus on $fi#ing !uickl'% than

$putting the correct fi# !uickl'%

Code reviews needs to be "ade

"andator' and should be planned forespeciall' for platfor"s like QCCo""on

+9 6=- etc. Currentl' there is littlebandwidth to be able to N

h, Co"ple#it' of the application not

alwa's taken into account whendeciding resources in a tea".

ince QCCo""on is a fra"ework that

shall i"pact "ultiple progra"s (11 for&pril0&ug ;) "ore e#perienced (at least

70@ 'ears) and welltrained developersshould be recruited for the tea".

h, 5he fact that the code isun"aintainable was often cited b'

the developers. =t was not provedusing "etrics and defects tracking.

Current code "etrics collection anddefects tracking proect "anage"ent

initiatives shows that CC "etrics ofQCCo""on e#ceeds the baseline for

"anageable code. 5he /efect data clearl'indicates that H;O of the defects which

are assigned to QCCo""on are assignedto +ther &pplications like RQ6=- after

anal'sis. 9ven tough these defects areresolved b' other tea"s QCCo""on

does the preli"inar' anal'sis and givesproper inputs to e#ternal application

tea"s.

5hese should affect decision "aking

about resource allocation too.

Page:11


12/18

Solutio!:

mmediate Solutio!s:

o ignificant a"ount and continuous refactoring of code is re!uired in

order to put functionalit' where it belongs and to break down the codeinto "ore "anageable pieces.

o & process driven approach for fi#ing defects should be adapted for

defect fi#ing. &ccurate i"pact anal'sis should be done before the

defects are actuall' fi#ed and checked in. Currentl' this is not possiblesince we have few resources offshore who can co"fortabl' work on

the fra"ework.

o 5raining the tea" on the whole Business process of +rder -anage"ent

will change the perspective and bring in "ore clarit' about the s'ste".

Lo!g 3erm Solutio!s:

o =nternal training on $Refectoring% "ethodologies (devised b' -artin

3owler and ent Beck) should beco"e part of regular training for all

developers. = have seen si"ilar proble"s in the code (although not tothe sa"e level) in 5>=? Quoting +& +9C and -/ applications.

=n QCCo""on we had an internal training b' rinivas udhindra onthe "ethodologies. &fter that we have partiall' refactored classes.

o 3or"alie code reviews. Code review te"plates refactoring plans and

tracking need to be created. Plan bandwidth for the sa"e.

o +ne of the factors in deciding on resource allocation should be whether

an application is going to affect "ultiple progra"s and has a possibilit'

to beco"e a bottleneck. 9#a"ple applications that fall into thatcategor' are: QCCo""on 6=- +9 6&5= e3low.

o

2.3.2. *oo mny dt $oints e!c+ned etween QOA nd QCCommon vi **/Request

%o!te-t:

QCCo""on and Q+& share the sa"e database (&693& for ales 3orce &uto"ation).QCCo""on paints the 9nd Point Configuration (9PC) or product configurator *= while

all other order entr' data points are entered using Q+& screens such as $Contacts%$6ocations% $Re!uest details% $+rder =te"% details etc.

=n 8etwor# 9PC *= is reached via Quoting and +rdering applications while in QCC/P 9PC is reached via Quoting +rdering and +9C applications.

ince QCCo""on is a fra"ework it doesn


13/18

5his leads to a brittle design where in $happ' path% scenarios even if a "inor changeis done on Q+& side QCCo""on 9PC *= is affected and defects start pouring in. =n

failure scenarios the proble" is even "ore co""on.

Prolem: *ther applicatio!s li4e $*A5 $uoti!g a!d *rderi!g share data

poi!ts #ith $%%ommo! through the re6uest o7ect or sometimes usi!g thesessio! o7ect 3hese data poi!ts asicall, co!trol the ehaviour o

h,perli!4s o! the (P% Page or U o the (P% page 3he data poi!ts shared ,

other applicatio!s are diere!t i! diere!t sce!arios A mi!or cha!ge toshared datapoi!ts #ill result i! a %ritical9 Sev 1 deect

Solutio!:

oi!g , asics o applicatio! i!tegratio!5 a rame#or4 a!d a! applicatio!

that use the rame#or4 should seldom e tightl, coupled 3he, must have acommo! poi!t to e-cha!ge data 3he, !eed to e loosel, coupled a!d data

poi!ts are e-cha!ged via a dataase or some sort o persiste!ce

mecha!ism

QCCo""on should receive onl' the "ini"u" para"eters re!uired and derive "ost

other para"eters fro" the /B. & factor' pattern can be used for differentiatinga"ong 8etwor# and /P specific data points.

/epending on database will ensure a robust i"ple"entation and prevention ofco""on defect on the *= in both $happ'path% and $% scenarios.

& si"ilar design was i"ple"ented so"e ti"e back b' Qwest Bangalore to preventleft navigation issues. Before the i"ple"entation $6eft8av% issues were !uite

co""on. &fter the i"ple"entation such issues have been al"ost nil.

2.#. *esti%ity

2.#.1. *estin vi 'Is nd c o /rormmtic4Automted *estin

%o!te-t:

QCCo""on has a testing application that dev tea" "e"bers utilie to test

validations and *=. o"e of the li"itations of the testing application are:

=t re!uires re!uest and base data to setup. =n affect onl' !uotes0orders

pree#isting in the database can be used for testing.

*= based testing leads to ti"e consu"ing debugging c'cle as even a si"ple

piece of code such as a >eb service invocation is often tested through the *=.

5he application re!uires RQ graphs (that QCCo""on uses to render0validate

order config data points) to be present

&lso due to coupling of presentation and do"ain la'er the testabilit' is ha"pered.

Prolem:

5here is no data setup "echanis" and progra""atic0auto"ated testing for "ost

functionalit' for the QCCo""on test application. &ll functionalit' is tested usingbrowserbased presentations.

Page:1@


14/18

&uto"ation tools that capture "ouse clicks and D55P re!uests are usuall' not ver'useful in a d'na"ic environ"ent because the resulting "acros are trick' to "aintain.

/riving auto"ated tests through direct function calls is far easier.

Solutio!:

Create co"prehensive data setup "echanis"s (invoking creating du""'

graphs stored procs publishing events etc. to create test data). 5his wastried out successfull' for co""on i"ple"entation of =PvH. 5he Product buildgraph was changed b' the Co""on /evelop"ent tea" to test the

i"ple"entation.

/evice testing "echanis"s such as *nit test suites (including test data

setup) to test "ost of the application. 5est coverage has to be high. *nit 5estcases were developed for testing the functions developed for e#panding and

abbreviating =P &ddress in =P4H for"at.

3. De$%oyment 0nvironment

3.1. De$%oyment issues

%o!te-t:

*pon Quoting0+rdering build QCCo""on code is also built and packaged along withthe Quoting0+rdering deplo'ables.

Q+& uses a nu"ber of bus and >eb services. 5he ?-6 sche"as are checked in andbuilt partl' with Q+& and partl' in QCCo""on. 5he reason it was done so earlier

was that QCCo""on was e#pected to have "ost of the co""on entities between

Quoting and +rdering. ?-6 bean obects are generated out of the ?-6 sche"as andused in Q+& code.

/uring the course of 8etwor# releases so"e of the environ"ent issues observedwere:

&fter deplo'"ent of +rdering0QCCo""on applications the server needs to be

bounced before testing can be done. *ne#pected errors occur if the path is

not followed. 5his is still observed to this date.

ars containing so"e of the ?-6 bean obects have to be loaded in a certain

order when the deplo'"ent happens. 5his speciall' beco"es an issue when a

new environ"ent is setup and so"eti"es we have spent da's tr'ing to get anenviron"ent up in dev =85 s'ste" test =54 end to end 979 environ"ents.

=n fact at different points in ti"e "aor functionalit' such as -ulti +rder 3ile upload

feature was blocked for "ore that a couple of weeks because it wasn


15/18

5he chief proble"s causing the issues "entioned above are:

?-6 bean obects in different ars interfere with each other

&s functionalit' was added in 8etwor# new classes were added to the sa"e

ars within the ear file without realiing the affect of the sa"e.

Solutio!:

5he solution for the sa"e is:

pecif'ing distinct na"espaces while generating ?-6 bean obects in both

Q+& and QCCo""on and using the" at the right locations

Break down the bigger ars in different ar units based on the appropriate

deplo'"ent structure and reference each other via $-anifest% files.

3.2. RQ r$+ enertion re%ted Issues uncovered too %te

%o!te-t:

&s "entioned earlier a QCCo""on batch application retrieves and stores a localcop' of product definition data and configuration ele"ents for efficient searches.5his process is co""onl' referred to as graph generation process. &t runti"e

QCCo""on paints the product configuration data points and validates user entriesbased on the graphs generated during graph generation process.

2raphs are generated using two "echanis"s:

Pulling out the latest product build ?-6s fro" RQ (Product Catalog). >hen

this is done version of the graph for the product in both &693& /B and the

deplo'"ent server is incre"ented b' one.

Pulling out the graphs stored in &693& i.e. pulling out the last saved graphs

in &693& rather than retrieving the latest graphs fro" product build0RQ.5'picall' Billing0Price i"pacting (&bove the 6ine or &56) graphs are pulled fro" RQin Quoting. &56 are pulled fro" the last saved version in &693& /B in +rdering.

Provisioning i"pacting graphs are pulled fro" RQ in +rdering.

=t was observed ver' fre!uentl' during 8etwor# and /P0Product releases that there

are issues during the graph generation process so"e of which are as follows:

-is"atch between the graph version in deplo'"ent environ"ent and the last

saved version no. in &693&

=ncorrect release tags or other para"eters specified b' Q+& in their propert'

files

Prolem:

=ssues related to graph generation for one0"ore products are discovered b' testersonl' when s'ste" testing tea"s tests their testcases. 5his leads to a large a"ount

of $9nviron"ent% related downti"es too "uch of co""unication betweendevelop"ent testing and test operations tea"s. = believe this was the "ost

fre!uentl' observed $9nviron"ent% issue.

Solutio!:

Page:1G


16/18

QCCo""on dev tea" should i"ple"ent appropriate checks and a consolidatedlogging at the end of product build generation process. 5est operations should look at

the logs and confir" if graph generation co"pletel' appropriatel'. +nl' if graphgeneration succeeds should Quoting0+rdering servers be brought up since "ost

test +rdering0Quoting test cases re!uire product configuration. =f proble" occursthe single point of contact should be QCCo""on dev0test tea"s.

5he above QCCo""on i"ple"entation and process change shall ensure faster

discover' of proble"s and resolution as well as reduced visibilit' of "'sterious$9nviron"ent% proble"s.

#. /roduct Deinition & RQ Dt

#.1. 'nit *estin o RQ Deinition

Prolem: Discrepe!cies i! Product ;uild Dei!itio! !eeds to e a!al,sed ,%ommo!

8o $fast turnaround% testing environ"ent. 8eed unit test capabilit'.

4alidation of decision para"eters doesn


17/18

Better tooling in the for" of bringing in capabilities in Product Builder (or analternate application) to validate consistenc' of RQ /ata vs. Rate Characteristics and

RC Ranges used for Pricing.

uch a 5ool could validate auto"aticall' that a RC or its RCRng defined to price a

8etwor#0/P i"ple"ented Billing Co"ponent "ap to RQ defined attributes and4alues for the sa"e Co"ponent.

o"e a"ount of validations is built in Product Builder but the' are often tuned off inBilling ervices.

%o!se6ue!ces:

ince validation of pricing RC and RCR82 would be done upfront these would not

be discovered at s'ste" testing ti"e. 5his would lead to reduced downti"es of the

s'ste" test environ"ent.

#.2.1. /oor *oo%in or 5iewin6 0ditin & 'nit *estin o RQ

#.3. Ot+er 7nown Issues

#.3.1. De$endency o /rovisionin 8or%ow (COR04e9O8) on COD $ssed yOrderin Systems or t+eir testin

Currentl' "ost of the testing in C+R90e3low is dependent on C+/s passed b'ordering s'ste"s. &ccording to unitha (Qwest Bangalore e3low lead) "ost

functionalit' is directl' tested in s'ste" test environ"ent. &lthough portions of devtesting happens using so"e a"ount of stubbing in the e3low code but "ost testing

happens directl' on s'ste" testing.

Progra" "ilestones t'picall' ensure e3low s'ste" test iterations follow +rder flowthroughs "ilestones for +rdering s'ste"s. 5his leads "ost provisioning and orderingdependencies related proble"s co"ing during the end of the release and too "an'

co"ple# defects increasing the risk on the progra"s.

5here should be a "echanis" for generating C+/s out of product builds that can be

used b' C+R90e3low.

Page:1I


18/18

:. Conc%usion

:.1. Reerences

1S /P &rchitecture A ="pl Principles A 2ood and bads A etc. A 7;;I;Ihttp:00!share0sites0!ccsdp0=5O7;/ocu"entation0&rchitecture0/PO7;&rchitecture

O7;O7;="plO7;principlesO7;O7;2oodsO7;andO7;badsO7;O7;etc.O7;O7;7;;I;I.doc

7S

Page:1
http://qshare/sites/qccsdp/IT%20Documentation/Architecture/SDP%20Architecture%20-%20Impl%20principles%20-%20Goods%20and%20bads%20-%20etc.%20-%20200707.dochttp://qshare/sites/qccsdp/IT%20Documentation/Architecture/SDP%20Architecture%20-%20Impl%20principles%20-%20Goods%20and%20bads%20-%20etc.%20-%20200707.dochttp://qshare/sites/qccsdp/IT%20Documentation/Architecture/SDP%20Architecture%20-%20Impl%20principles%20-%20Goods%20and%20bads%20-%20etc.%20-%20200707.dochttp://qshare/sites/qccsdp/IT%20Documentation/Architecture/SDP%20Architecture%20-%20Impl%20principles%20-%20Goods%20and%20bads%20-%20etc.%20-%20200707.dochttp://qshare/sites/qccsdp/IT%20Documentation/Architecture/SDP%20Architecture%20-%20Impl%20principles%20-%20Goods%20and%20bads%20-%20etc.%20-%20200707.dochttp://qshare/sites/qccsdp/IT%20Documentation/Architecture/SDP%20Architecture%20-%20Impl%20principles%20-%20Goods%20and%20bads%20-%20etc.%20-%20200707.doc

qccommon rq

Documents