Download - Test Strategy Utilising Mc Useful Tools
High Level Test Strategy overview
Test Strategy
• “Layer” the project
• Identify the components in each “layer”
• Identify the test basis
• Draw up a Dependency matrix
• Formulate a testing “Big Picture”
• Test Packs
• Summarising and going forward ….
High Level Test Strategy overview
“Layer” the project
• Integration services runs on (currently) 3 Biztalk platforms: IS2000 (PoS), IS2002 (ISGM) and IS2004
• Each platform contains various abstract layers:– Service layer– Application layer– Database layer– Biztalk layer
Queue
ReceiveFunctions
Adapters
Orchestrations
MessageBoxes
ServerHosts– Test Environment
High Level Test Strategy overview
Components in each layer
• Service layer– IS01, IS02, IS03, IS04, BP01, BP02 etc.
• Application layer– RuleExec, Janus Web, HOPPA etc.
• Database layer– NDTRefData, Alerter, HAT etc.
• Biztalk layer– Queue - Workqueue, Retry, Scheduled etc.– ReceiveFunctions - Failed Messages, Receiver– Adapters - FTP, HTTP, SOAP etc.– Orchestrations – BP01, BP02 etc.– MessageBoxes – MsgBox DB, BAM archive etc.– ServerHosts – Orchestration host, Tracking host, SendHost adapters etc.
• Test Environment– DEV, Systest 1, Systest 2, CATE 1, CATE 2, little CATE etc.
High Level Test Strategy overview
Test basis N
• This is the documentation that supplies the Expectations, Requirements and Expected Results for ANY given piece of testing effort
• Prime source of these here is the SUMMIT methodology deliverables – i.e. Project Charter, System Prospectus, System Delivery Spec (SDS), Tech System Design (TSD), Tech Procedure Development (TPD) etc.
• Test basis should (ideally) consist of SIGNED-OFF DOCUMENTATION ONLY. However – real-world here and it may include draft docs but increased risk of inaccurate Requirements and/or Expected Results
• All pertinent test basis documents collated as “KeyDocs” and are secured into Quality Center
High Level Test Strategy overview
Dependency matrix N
• This matrix will highlight the implicit and explicit relationship between IS components on each abstract layer
• It is used for impact analysis and, hence, to identify likely candidates for regression testing here in IS
• Where change is known to have occurred to one component in the abstract layer model, the reader can quickly follow the implicit and explicit relationships to other components that are identified in this matrix. As the reader navigates through the matrix, further likely regression test candidates (and hence Test Packs) are identified
• Change to one particular component may involve regression testing “most” of IS
High Level Test Strategy overview
Testing Big Picture N
• This is, in essence, a amalgamation of all of the useful diagrams identified as part of the test basis
• This model offers a tabular view of the testing and will involve the decomposition of those diagrams in the KeyDocs repository
• Data from the diagrams is dropped into a two-dimensional table and the information from the diagram (Names in boxes, names against lines, where the lines come from, where the lines go to etc) is then accessible AND, more importantly, reusable from a testing perspective
• Impact analysis is greatly facilitated by using this Big Picture…You can not only prove the Dependency matrix – which means you can accurately describe what related, regression tests need to be run but you also have a pictorial view of everything testing. INVALUABLE
High Level Test Strategy overview
Test Packs N
• The management will usually have a view at what level of granularity a Test Packs should be designed. Integration Services (IS) have chosen “layers” within platform. i.e ISGM Biztalk, or IS2004 Database, or IS01 service etc.
• The FEWER Test Packs you choose to have, the MORE has to go into each Test Pack. This increases complexity, which increases the chance of failure in a Test Pack, BUT the MORE Test Packs you have, the more resources required to create, modify, prepare and execute them!
• If you “spread the load” by having more Test Packs (higher level of granularity), then you have a more agile way of testing change BECAUSE you wont need to regression test ALL Test Packs.
High Level Test Strategy overview
Summarising N
• Test Packs are used to regression test• KeyDocs repository contains the (signed-off and master) test basis documentation which
will facilitate proof of test coverage (i.e. HAVE we covered everything in the documentation?)• Dependency matrix helps you to decide (accurately when used in conjunction with the
Big Picture) exactly what else is candidate for test• Big Picture gives pictorial representation of ALL things testing on a project/programme.
The data/information contained therein is ultimately reusable because it also exists in two-dimensional form!
• If all this seems VERY COMPLEX, then wait till something CHANGES. i.e. documentation or process. THEN it gets complex because you don’t know what impact that change will have!!
• The beauty of the above is that I have tools that will automatically decompose documents/diagrams so “change” merely means re-extraction of the data from the changed documents or diagrams (text from documents or text/context information from diagrams) into KeyDocs or Big Picture……. Next slide elaborates more
• ….. But this is all talking the talk, now lets walk the walk
High Level Test Strategy overview
Walk the walk N
• DocIndex provides the engine that drives the creation of the KeyDocs (test coverage and Expected Results).
• InternetMiner provides the engine that assists the creation of KeyDocs and operates on Intranet/internet pages and hyperlinks to extract similar information to DocIndex. This is more an ancillary tool that complements DocIndex and is useful for gaining insight into specific areas of expertise.
• VisioDecompositer provides the engine that drives diagram information (process flow and data flow) into the Big Picture.
• Here’s some screen shots to give you a better flavour….
High Level Test Strategy overview
DocIndex 1 of 10N
• DocIndex (viewer) main menu
High Level Test Strategy overview
DocIndex 2 of 10
• KeyDocs main menu
High Level Test Strategy overview
DocIndex 3 of 10
• Display a selectable list of KeyDocs
High Level Test Strategy overview
DocIndex 4 of 10
• Now display a summarised list of the sections in the selected KeyDoc
High Level Test Strategy overview
DocIndex 5 of 10
• Select one of the sections in the document and the text is displayed
High Level Test Strategy overview
DocIndex 6 of 10
• The power of this tool lies in the fact that ALL relevant project/programme-level documentation is captured into the tool
High Level Test Strategy overview
DocIndex 7 of 10
• The contents of all of the KeyDocs are stored in two tables: tblSectionHeaders – which is a summarised “table of contents” for a KeyDoc
High Level Test Strategy overview
DocIndex 8 of 10
• and File_Decomposition – which is textual body for all KeyDocs
High Level Test Strategy overview
DocIndex 9 of 10
• There is a keyword/key phrase Search facility for the group of KeyDocs, which allows the user to quickly become acquainted with “project specific” terminology
High Level Test Strategy overview
DocIndex 10 of 10
• and the reader can, very quickly, become acquainted with the context in which the specific search terms are used.
High Level Test Strategy overview
InternetMiner 1 of 10 N
• InternetMiner main menu
High Level Test Strategy overview
InternetMiner 2 of 10
• Enter a suitably relevant URL - in this case, found by doing a “biztalk performance test” search string in Google UK
High Level Test Strategy overview
InternetMiner 3 of 10
• And the page(s) are decomposed into InternetMiner, in much the same way as DocIndex
High Level Test Strategy overview
InternetMiner 4 of 10
• The list of decomposed URLs can be viewed (and refreshed either manually or automatically) so that the information contained therein is always up to date
High Level Test Strategy overview
InternetMiner 5 of 10
• And the content from ALL of the decomposed URLs can be viewed at a glance
High Level Test Strategy overview
InternetMiner 6 of 10
• And relevant search keywords or key phrases can be entered
High Level Test Strategy overview
InternetMiner 7 of 10
• And the result be displayed (in various forms) - so that the user can see, contextually, where the search term is used
High Level Test Strategy overview
InternetMiner 8 of 10
• Alternative views of the results (and search strings) can be made
High Level Test Strategy overview
InternetMiner 9 of 10
• And you can quickly build up a picture of the various (in this case technical Biztalk) phrases
High Level Test Strategy overview
InternetMiner 10 of 10
• And how they relate to the overall topic (Biztalk Performance testing)
High Level Test Strategy overview
VisioDecompositer 1 of 2N
• VisioDecompositer takes a straightforward Visio diagram…..
High Level Test Strategy overview
VisioDecompositer 2 of 2N
• And converts it into two-dimensional information that can be “mapped” into test cases or test scripts quite easily