achieve competitive advantage with ibm devops · rational functional tester •record, enhance, and...
TRANSCRIPT
Rational SoftwareRational SoftwareRational SoftwareRational Software
Achieve Competitive Advantage with IBM DevOps
Quality is not Accidental
IBM Competitive Project Office© 2014 IBM Corporation
Rational SoftwareRational SoftwareRational SoftwareRational Software
Introduction
Break Down the Barriers with Improved Planning and Collaboration
Define, Manage, and Track your Requirements and Designs
Agenda
© 2014 IBM CorporationIBM Competitive Project Office 2
Integrated Change, Build and Deploy is What Sets Us Apart
Quality is not Accidental
Bridge the Gap between Development and Operations
Put your Business in Motion - Become a Mobile Enterprise
Rational SoftwareRational SoftwareRational SoftwareRational Software
Errors caught early cost less to fix . . .
“30%30%30%30% of project costs are due to rework and poor execution of requirements”
“50%50%50%50% of outsourced projects are expected to under perform”
“34%34%34%34% of software projects are deemed successful costing $300B annually”
• Bugs/failures are exponentially more expensive the longer they go uncaught
• Extends to analysis, requirements, development, and everywhere across the application lifecycle
© 2014 IBM CorporationIBM Competitive Project Office 3
Rela
tive
Co
st t
o R
ep
air
AcceptanceUnit TestCoding Maintenance
1.5x
Stage in which Error is Discovered
and poor execution of requirements” to under perform”
Analysis Design
5x10x
20x
50x
200x
Rational SoftwareRational SoftwareRational SoftwareRational Software
Quality management must span the lifecycle
Release to ProductionTesting
Acceptance
System
Performance
Analysis
Requirements
Main
ten
an
ce
© 2014 IBM CorporationIBM Competitive Project Office 4
Integration Testing
Unit Testing
Functional Testing
Coding
Main
ten
an
ce
All of these tests must be planned. All of these tests must be planned. All of these tests must be planned. All of these tests must be planned. Artifacts Artifacts Artifacts Artifacts created by tests need to be managed, created by tests need to be managed, created by tests need to be managed, created by tests need to be managed, linked to other development artifacts, and linked to other development artifacts, and linked to other development artifacts, and linked to other development artifacts, and analyzed.analyzed.analyzed.analyzed.
Rational SoftwareRational SoftwareRational SoftwareRational Software
Definition of common test types
• Unit testing – Test individual units of source code
– Determines if code works as intended (fit for use)
• Functional testing– Ignores the internal mechanism of a system or component
– Focuses solely on the outputs generated in response to selected inputs and execution conditions
– Can be manual or automated
© 2014 IBM CorporationIBM Competitive Project Office 5
– Can be manual or automated
• Performance testing– Evaluates the compliance of a system or component with specified
performance requirements
• Integration testing– Input: modules that have been unit tested
– Groups modules in larger aggregates and applies tests defined in an integration test plan
– Output: integrated system ready for system testing
Rational SoftwareRational SoftwareRational SoftwareRational Software
Rational Quality Manager provides a management hub for quality artifacts
RQM Client: Web browser
Test Planning:
Comprehensive test plan,
Shared objectives, Scope,
Timeline, Resources, Risk
assessment Test Execution:
Manual test execution, Use
test automation tools, Record
test results, Submit & track
defectsReporting & Dashboard:
Status & progress tracking,
Customizable live
Test Construction:
Requirement-driven testing,
Test environments
coverage, Manual test
authoring, Test lab
management
Rational DOORS Next Generation
Rational DOORS Next Generation
Collections
Requirements
Rational Software
Architect Design Manager
Rational Software
Architect Design Manager
Designs
Models
Rational Team Concert
Rational Team Concert
Release Plans
Iterations
© 2014 IBM CorporationIBM Competitive Project Office 6
Jazz Team Server
RTCRDNG RQM
Core OSLC REST Services: Discovery, Query, Storage, Presentation, Data Warehousing,
Administration, Reporting, etc…
RDNG REST
Services: Rqmts
Management
RTC REST
Services: Change
Management
RQM REST
Services: Quality
Management
RSA DM
Customizable live
dashboard, Real-time
metrics and reports,
Compliance and quality
audit
Team Collaboration:
Process enactment and
enforcement, reviews, task
management, Rapid team
member on-boarding
Storyboards
Use cases
…
Diagrams
…
Work Items
Source Code
…
RSA DM REST
Services: Design
Management
Rational SoftwareRational SoftwareRational SoftwareRational Software
Rational Quality Manager – test planningThe test plan is the core artifact that
contains all information pertaining to
the quality management effort
RQM Client: Web browser
Test Planning:
Comprehensive test plan,
Shared objectives, Scope,
Timeline, Resources, Risk
assessment Test Execution:
Manual test execution, Use
test automation tools, Record
test results, Submit & track
defectsReporting & Dashboard:
Status & progress tracking,
Customizable live
Test Construction:
Requirement-driven testing,
Test environments
coverage, Manual test
authoring, Test lab
management
Team Collaboration:
Rational DOORS Next Generation
Rational DOORS Next Generation
Collections
Requirements
Storyboards
Rational Software
Architect Design Manager
Rational Software
Architect Design Manager
Designs
Models
Rational Team Concert
Rational Team Concert
Release Plans
Iterations
Work Items
© 2014 IBM CorporationIBM Competitive Project Office 7
Jazz Team Server
RTCRDNG RQM
Core OSLC REST Services: Discovery, Query, Storage, Presentation, Data Warehousing,
Administration, Reporting, etc…
RDNG REST
Services: Rqmts
Management
RTC REST
Services: Change
Management
RQM REST
Services: Quality
Management
RSA DM
Customizable live
dashboard, Real-time
metrics and reports,
Compliance and quality
audit
Team Collaboration:
Process enactment and
enforcement, reviews, task
management, Rapid team
member on-boarding
Storyboards
Use cases
…
Diagrams
…
Work Items
Source Code
…
RSA DM REST
Services: Design
Management
Rational SoftwareRational SoftwareRational SoftwareRational Software
• Central location for business objectives, requirements, resources, platform and exit criteria to name a few
• Defined Responsibilities
Individual sections
• Goal-Oriented
Formalized and
Rational Quality Manager’s test plan
© 2014 IBM CorporationIBM Competitive Project Office 8
Formalized and documented exit criteria
• Extensible
Add sections, import custom data
• Snapshot version control to track plan history
Rational SoftwareRational SoftwareRational SoftwareRational Software
• Planning fields for test artifacts to help prioritize, schedule and manage testing activities
Test prioritization helpers
• Test Suite/Case Execution Record section in test plans
Traceability and execution progress
views
Comprehensive test plan
Unify the team through real-time collaboration and clear objectives
© 2014 IBM CorporationIBM Competitive Project Office 9
Rational SoftwareRational SoftwareRational SoftwareRational Software
Rational Quality Manager – test constructionRQM has built-in function to allow teams
to track lab assets and configurations, provision lab assets, and schedule and reserve the assets
RQM Client: Web browser
Test Planning:
Comprehensive test plan,
Shared objectives, Scope,
Timeline, Resources, Risk
assessment Test Execution:
Manual test execution, Use
test automation tools, Record
test results, Submit & track
defectsReporting & Dashboard:
Status & progress tracking,
Customizable live
Test Construction:
Requirement-driven testing,
Test environments
coverage, Manual test
authoring, Test lab
management
Team Collaboration:
Rational DOORS Next Generation
Rational DOORS Next Generation
Collections
Requirements
Storyboards
Rational Software
Architect Design Manager
Rational Software
Architect Design Manager
Designs
Models
Rational Team Concert
Rational Team Concert
Release Plans
Iterations
Work Items
© 2014 IBM CorporationIBM Competitive Project Office 10
Jazz Team Server
RTCRDNG RQM
Core OSLC REST Services: Discovery, Query, Storage, Presentation, Data Warehousing,
Administration, Reporting, etc…
RDNG REST
Services: Rqmts
Management
RTC REST
Services: Change
Management
RQM REST
Services: Quality
Management
RSA DM
Customizable live
dashboard, Real-time
metrics and reports,
Compliance and quality
audit
Team Collaboration:
Process enactment and
enforcement, reviews, task
management, Rapid team
member on-boarding
Storyboards
Use cases
…
Diagrams
…
Work Items
Source Code
…
RSA DM REST
Services: Design
Management
Rational SoftwareRational SoftwareRational SoftwareRational Software
• Requirements management and integration
Requirements-driven testing with RQM
New and New and New and New and UpdatedUpdatedUpdatedUpdated
RequirementsRequirementsRequirementsRequirements
Requirement Requirement Requirement Requirement update notificationupdate notificationupdate notificationupdate notification
Duplicate Duplicate Duplicate Duplicate RequirementRequirementRequirementRequirementPreventionPreventionPreventionPrevention
Requirement Requirement Requirement Requirement impact analysisimpact analysisimpact analysisimpact analysis
Dynamic Dynamic Dynamic Dynamic Test PlanTest PlanTest PlanTest Plan
© 2014 IBM CorporationIBM Competitive Project Office 11
Requirements management and integration
• Advanced requirement analysis prevents duplicates and duplicate efforts
• Review and approvals of tasks across projects
• Asset versioning with dynamic update
• Define requirements in rich text
Review & Review & Review & Review & ApproveApproveApproveApprove
Rational SoftwareRational SoftwareRational SoftwareRational Software
• Create test scripts manually or automatically• Record user interactions with an application under test using RFT• Recording is captured as a RQM manual test script in English natural
language with screenshots, independent from RFT• Resulting manual test scripts are executed manually using RQM only• Supports Web and Java applications running on Windows
Author manual test scripts step-by-step or record them with IBM Rational Functional Tester
© 2014 IBM CorporationIBM Competitive Project Office 12
Accelerate and streamline manual
testing creation
Rational SoftwareRational SoftwareRational SoftwareRational Software
• RQM includes test lab management capability
– Capture and organize test lab inventory
– Manage requests and reservations of test lab components
– Deploy test tools to lab machines
Analyze usage patterns to optimize lab utilization
Built-in test lab management
© 2014 IBM CorporationIBM Competitive Project Office 13
– Analyze usage patterns to optimize lab utilization
– Do comprehensive reporting on test assets
• Integrates with virtual automation platforms, e.g., Surgient and VMLogix
• RQM can integrate with 3rd party test lab management products
Rational SoftwareRational SoftwareRational SoftwareRational Software
• Manage
– Verify that JKE has the resources required to fulfill their test plan
• Deploy
– Deliver the configurations JKE’s
JKE gains more test time
40% Lab Setup40% Lab Setup 60% Test Time60% Test TimeBefore lab automation
Non-automated tasks or steps
© 2014 IBM CorporationIBM Competitive Project Office 14
configurations JKE’s teams require for test
• Optimize
– Analyze patterns to minimize cost and maximize utilization
Work smarter, save on test lab overhead, infrastructure and duration costs
60% Test Time60% Test Time
automation
With lab automation
Application setup with RQM (50% reduction)
20% Setup20%
Setup20% Free20% Free
Rational SoftwareRational SoftwareRational SoftwareRational Software
Rational Quality Manager – test executionRQM provides RQM provides RQM provides RQM provides builtbuiltbuiltbuilt----in in in in manual manual manual manual test authoring, test authoring, test authoring, test authoring, , execution, and management along with , execution, and management along with , execution, and management along with , execution, and management along with automated testing from various sourcesautomated testing from various sourcesautomated testing from various sourcesautomated testing from various sources
RQM Client: Web browser
Test Planning:
Comprehensive test plan,
Shared objectives, Scope,
Timeline, Resources, Risk
assessment Test Execution:
Manual test execution, Use
test automation tools, Record
test results, Submit & track
defectsReporting & Dashboard:
Status & progress tracking,
Customizable live
Test Construction:
Requirement-driven testing,
Test environments
coverage, Manual test
authoring, Test lab
management
Team Collaboration:
Rational DOORS Next Generation
Rational DOORS Next Generation
Collections
Requirements
Storyboards
Rational Software
Architect Design Manager
Rational Software
Architect Design Manager
Designs
Models
Rational Team Concert
Rational Team Concert
Release Plans
Iterations
Work Items
© 2014 IBM CorporationIBM Competitive Project Office 15
Jazz Team Server
RTCRDNG RQM
Core OSLC REST Services: Discovery, Query, Storage, Presentation, Data Warehousing,
Administration, Reporting, etc…
RDNG REST
Services: Rqmts
Management
RTC REST
Services: Change
Management
RQM REST
Services: Quality
Management
RSA DM
Customizable live
dashboard, Real-time
metrics and reports,
Compliance and quality
audit
Team Collaboration:
Process enactment and
enforcement, reviews, task
management, Rapid team
member on-boarding
Storyboards
Use cases
…
Diagrams
…
Work Items
Source Code
…
RSA DM REST
Services: Design
Management
Rational SoftwareRational SoftwareRational SoftwareRational Software
JKE’s manual testers can’t keep pace with the rapid releases of JKE’s agile development process
They need a tool to provide assistance to the manual tasks involved with manual testing (i.e. data entry, data validation, reuse)
JKE can’t keep up with their manual tests –they need tool automation, reuse, and management
Manual testing Manual testing Manual testing Manual testing
© 2014 IBM CorporationIBM Competitive Project Office 16
Manual testing Manual testing Manual testing Manual testing can’t keep pacecan’t keep pacecan’t keep pacecan’t keep pace!!!!
They must They must They must They must automate their automate their automate their automate their manual testing manual testing manual testing manual testing process!process!process!process!
TimeTimeTimeTime
Tests
Tests
Tests
Tests
Build 1Build 1Build 1Build 1
...take ...take ...take ...take more timemore timemore timemore time
...the new ...the new ...the new ...the new
tests...tests...tests...tests...
Build 2Build 2Build 2Build 2 Build 3, 4, 5 …Build 3, 4, 5 …Build 3, 4, 5 …Build 3, 4, 5 …
Rerun the first Rerun the first Rerun the first Rerun the first tests and...tests and...tests and...tests and...
Rational SoftwareRational SoftwareRational SoftwareRational Software
RQM’s manual tester assists manual test case executionRich Text Editor
Step by step capture and execution
Reusable keywords
© 2014 IBM CorporationIBM Competitive Project Office 17
Assisted data entry and validation
Automatic defect creation – send directly to RTC (or your own CCM tool)
Rational SoftwareRational SoftwareRational SoftwareRational Software
Use a mobile app for off-line test execution• Beneficial for testing actual products or systems while being in the field or in an
isolated test lab without connectivity
• Mobile application allows tests to be run while disconnected capturing verdicts and results
© 2014 IBM CorporationIBM Competitive Project Office 18
Connect app to server and download test scripts
Perform script steps and enter results (pass/fail, comments, pictures)
Complete steps and upload test record to server when device has connectivity
Rational SoftwareRational SoftwareRational SoftwareRational Software
Rational also offers a complete suite of testing tools
10101010
Rational Functional Tester
Rational Functional Tester
• Record, enhance, and execute automated tests
• Java and/or Visual Studio generated test scripts
• Record, enhance, and execute automated tests
• Java and/or Visual Studio generated test scripts
Rational Integration Tester
Rational Integration Tester
• Web service visual editors• WS-BPEL testing• Code-free testing
• Web service visual editors• WS-BPEL testing• Code-free testing
Rational Performance Tester
Rational Performance Tester
• Root cause analysis• Performance loads• Automated script capture
and playback
• Root cause analysis• Performance loads• Automated script capture
and playback
Adapters
Rational Test WorkbenchRational Test WorkbenchRational Test WorkbenchRational Test Workbench
© 2014 IBM CorporationIBM Competitive Project Office 19
Jazz Team Server
Test Planning:Test Planning:Test Planning:Test Planning:Comprehensive test plan, Shared objectives, Scope, Comprehensive test plan, Shared objectives, Scope, Comprehensive test plan, Shared objectives, Scope, Comprehensive test plan, Shared objectives, Scope,
Timeline, Resources, Risk assessment Timeline, Resources, Risk assessment Timeline, Resources, Risk assessment Timeline, Resources, Risk assessment
Test Planning:Test Planning:Test Planning:Test Planning:Comprehensive test plan, Shared objectives, Scope, Comprehensive test plan, Shared objectives, Scope, Comprehensive test plan, Shared objectives, Scope, Comprehensive test plan, Shared objectives, Scope,
Timeline, Resources, Risk assessment Timeline, Resources, Risk assessment Timeline, Resources, Risk assessment Timeline, Resources, Risk assessment
Test Execution:Test Execution:Test Execution:Test Execution:Manual test execution, Use test automation tools, Record Manual test execution, Use test automation tools, Record Manual test execution, Use test automation tools, Record Manual test execution, Use test automation tools, Record
test results, Submit & track defectstest results, Submit & track defectstest results, Submit & track defectstest results, Submit & track defects
Test Execution:Test Execution:Test Execution:Test Execution:Manual test execution, Use test automation tools, Record Manual test execution, Use test automation tools, Record Manual test execution, Use test automation tools, Record Manual test execution, Use test automation tools, Record
test results, Submit & track defectstest results, Submit & track defectstest results, Submit & track defectstest results, Submit & track defects Reporting & Dashboard:Reporting & Dashboard:Reporting & Dashboard:Reporting & Dashboard:Status & progress tracking, Customizable live dashboard, Status & progress tracking, Customizable live dashboard, Status & progress tracking, Customizable live dashboard, Status & progress tracking, Customizable live dashboard, RealRealRealReal----time metrics and reports, Compliance and quality time metrics and reports, Compliance and quality time metrics and reports, Compliance and quality time metrics and reports, Compliance and quality
auditauditauditaudit
Reporting & Dashboard:Reporting & Dashboard:Reporting & Dashboard:Reporting & Dashboard:Status & progress tracking, Customizable live dashboard, Status & progress tracking, Customizable live dashboard, Status & progress tracking, Customizable live dashboard, Status & progress tracking, Customizable live dashboard, RealRealRealReal----time metrics and reports, Compliance and quality time metrics and reports, Compliance and quality time metrics and reports, Compliance and quality time metrics and reports, Compliance and quality
auditauditauditaudit
Test Construction:Test Construction:Test Construction:Test Construction:RequirementRequirementRequirementRequirement----driven testing, Test environments coverage, driven testing, Test environments coverage, driven testing, Test environments coverage, driven testing, Test environments coverage,
Manual test authoring, Test lab managementManual test authoring, Test lab managementManual test authoring, Test lab managementManual test authoring, Test lab management
Test Construction:Test Construction:Test Construction:Test Construction:RequirementRequirementRequirementRequirement----driven testing, Test environments coverage, driven testing, Test environments coverage, driven testing, Test environments coverage, driven testing, Test environments coverage,
Manual test authoring, Test lab managementManual test authoring, Test lab managementManual test authoring, Test lab managementManual test authoring, Test lab management
Team Collaboration:Team Collaboration:Team Collaboration:Team Collaboration:Process enactment and enforcement, reviews, task Process enactment and enforcement, reviews, task Process enactment and enforcement, reviews, task Process enactment and enforcement, reviews, task management, Rapid team member onmanagement, Rapid team member onmanagement, Rapid team member onmanagement, Rapid team member on----boardingboardingboardingboarding
Team Collaboration:Team Collaboration:Team Collaboration:Team Collaboration:Process enactment and enforcement, reviews, task Process enactment and enforcement, reviews, task Process enactment and enforcement, reviews, task Process enactment and enforcement, reviews, task management, Rapid team member onmanagement, Rapid team member onmanagement, Rapid team member onmanagement, Rapid team member on----boardingboardingboardingboarding
Rational Quality ManagerRational Quality ManagerRational Quality ManagerRational Quality Manager
Rational SoftwareRational SoftwareRational SoftwareRational Software
RQM + RTW (RFT + RPT + RIT) + RTVS = increased reuse and centrally RQM + RTW (RFT + RPT + RIT) + RTVS = increased reuse and centrally RQM + RTW (RFT + RPT + RIT) + RTVS = increased reuse and centrally RQM + RTW (RFT + RPT + RIT) + RTVS = increased reuse and centrally managed Automated Functional, Performance and managed Automated Functional, Performance and managed Automated Functional, Performance and managed Automated Functional, Performance and Integration/Virtualization TestingIntegration/Virtualization TestingIntegration/Virtualization TestingIntegration/Virtualization Testing
• Rational Quality Manager provides complete lifecycle integration
– Automatically populate defects, scripts, results, etc.
– Testers don’t need to learn how to use RQM – their artifacts are reused
RQM integration with Rational testing tools extends quality management
© 2014 IBM CorporationIBM Competitive Project Office 20
– Testers don’t need to learn how to use RQM – their artifacts are reused from RQM that connects via adapters to the testing tools
– All artifacts can be linked to RDNG and RTC from RQM for greater traceability
• Build and execute test cases with scripts from any combination of testing tools
– Rational Functional Tester, Rational Performance Tester, Rational Integration Tester, and Rational Test Virtualization Server
– Example: RFT or RIT scripts combined in a performance test case
– Example: RTVS stub deployed and started from RQM
Rational SoftwareRational SoftwareRational SoftwareRational Software
Definition: Definition: Definition: Definition: Testing that ignores the internal mechanism of a system or component and focuses solely on the outputs generated in response to selected inputs and execution conditions.1
• Automated test scripts are more productive
• Use to test new code or for regression testing on changed code
Automated functional testing
© 2014 IBM CorporationIBM Competitive Project Office 21
1[IEEE 90] Institute of Electrical and Electronics Engineers. IEEE Standard Computer Dictionary: A Compilation of IEEE
Standard Computer Glossaries. New York, NY: 1990.
Record
Test ScriptEnhance
Recorded Script
Execute
Test Scripts
Rational SoftwareRational SoftwareRational SoftwareRational Software
• Use Rational Functional Tester
Record/Enhance/Execute scripts on Windows/Linux
Functional test any .NET, Web, Java application, Siebel, SAP, and many more
Rational Functional Tester
.NET, Browser, Java UI
© 2014 IBM CorporationIBM Competitive Project Office 22
• Use Rational Functional Tester Extension for Terminal-based Applications
Record/Enhance/Execute scripts on Windows
Functional test System z terminal based applications
System z Terminal UI
Rational SoftwareRational SoftwareRational SoftwareRational Software
Definition: Testing conducted to evaluate the compliance of a system or component with specified performance requirements1
The evaluation of a system’s capability to provide acceptable response times and function while serving a large number of clients
Also known as Load Testing, Capacity Testing, or Stress Testing
Performance testing
16 hits/sec
Page hit rate
© 2014 IBM CorporationIBM Competitive Project Office 23
1[IEEE 90] Institute of Electrical and Electronics Engineers. IEEE Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries. New York, NY: 1990.
12 hits/sec
Page hit rate
Time
Increase user load
Died Died Died Died with with with with 700 700 700 700 usersusersusersusers
700
16 hits/sec
Time
Increase user load2,000
Well Well Well Well behaved behaved behaved behaved up to up to up to up to 2,000 users2,000 users2,000 users2,000 users
Rational SoftwareRational SoftwareRational SoftwareRational Software
Use Rational Performance Tester to:
• Develop and execute scripts on Windows or Linux
• Collect server resource data and application performance data in the same product
Rational Performance Tester
Any Web Application
© 2014 IBM CorporationIBM Competitive Project Office 24
performance data in the same product
• No programming knowledge necessary to create/execute test
Perform tests on a variety of applications
Root Cause Analysis – identify both source code and application tier causing the bottleneck
Rational SoftwareRational SoftwareRational SoftwareRational Software
DEMO 1: Integration between RQM and RFT
He plays the script back and watches the behavior of the
Tanuj creates an automated functional test with Rational Functional Tester recorder
© 2014 IBM CorporationIBM Competitive Project Office 25
He analyses the test results to check for any errors or defects
He plays the script back and watches the behavior of the application
Rational SoftwareRational SoftwareRational SoftwareRational Software
We’ve encountered We’ve encountered We’ve encountered We’ve encountered some challenges some challenges some challenges some challenges as our IT environment as our IT environment as our IT environment as our IT environment has become has become has become has become more more more more complex and the development complex and the development complex and the development complex and the development teams deliver teams deliver teams deliver teams deliver more changes more changes more changes more changes faster. faster. faster. faster. Do you have a solution that could Do you have a solution that could Do you have a solution that could Do you have a solution that could help us accelerate integration help us accelerate integration help us accelerate integration help us accelerate integration testing?testing?testing?testing?
Yes, we do Yes, we do Yes, we do Yes, we do –––– Rational Test Rational Test Rational Test Rational Test Workbench and Servers (Green Workbench and Servers (Green Workbench and Servers (Green Workbench and Servers (Green Hat) help significantly reduce Hat) help significantly reduce Hat) help significantly reduce Hat) help significantly reduce testing time and improve quality testing time and improve quality testing time and improve quality testing time and improve quality with its innovative test with its innovative test with its innovative test with its innovative test virtualization.virtualization.virtualization.virtualization.
© 2014 IBM CorporationIBM Competitive Project Office 26
IBMQA Manager
Rational SoftwareRational SoftwareRational SoftwareRational Software
Rational Test Workbench (RTW) and ServersAddress three major challenges in quality managementAddress three major challenges in quality managementAddress three major challenges in quality managementAddress three major challenges in quality management
Development Agility Development Agility Development Agility Development Agility and Speedand Speedand Speedand Speed
Increasing Increasing Increasing Increasing Cost of QualityCost of QualityCost of QualityCost of Quality
Increasing Increasing Increasing Increasing Development ComplexityDevelopment ComplexityDevelopment ComplexityDevelopment Complexity
Public CloudPublic CloudPublic CloudPublic CloudPrivate CloudPrivate CloudPrivate CloudPrivate Cloud
CollaborationCollaborationCollaborationCollaboration
Web/Web/Web/Web/InternetInternetInternetInternet
Routing Routing Routing Routing ServiceServiceServiceService ContentContentContentContent
ProvidersProvidersProvidersProvidersEJBEJBEJBEJB
SharedSharedSharedShared
Business Business Business Business PartnersPartnersPartnersPartners
TraditionalTraditionalTraditionalTraditionalTesting Can’t Keep Testing Can’t Keep Testing Can’t Keep Testing Can’t Keep
up with Agile up with Agile up with Agile up with Agile VelocityVelocityVelocityVelocity
© 2014 IBM CorporationIBM Competitive Project Office 27
* Source:http://www.sei.cmu.edu/about/message/
Heterogeneous EnvironmentsHeterogeneous EnvironmentsHeterogeneous EnvironmentsHeterogeneous Environments
Data WarehouseData WarehouseData WarehouseData Warehouse MainframeMainframeMainframeMainframeEnterprise Enterprise Enterprise Enterprise Service BusService BusService BusService Bus
Directory Directory Directory Directory IdentityIdentityIdentityIdentity
File File File File systemssystemssystemssystems
ThirdThirdThirdThird----partypartypartypartyServicesServicesServicesServices PortalsPortalsPortalsPortals
SharedSharedSharedSharedServicesServicesServicesServices
ArchivesArchivesArchivesArchives
Messaging Messaging Messaging Messaging ServicesServicesServicesServices
Other costsOther costsOther costsOther costs
Testing costsTesting costsTesting costsTesting costs
Rational SoftwareRational SoftwareRational SoftwareRational Software
Traditional process involves the “big bang” integration testing after unit tests
System Tests
Requirements
Unit TestsBig Bang
© 2014 IBM CorporationIBM Competitive Project Office 28
UAT
OAT
Performance
Legend: quality gate
• Multiple defects introduced at once
• More unknowns – greater risk
• Result: more re-work, more expensive
Rational SoftwareRational SoftwareRational SoftwareRational Software
With Rational Test Virtualization Server and Rational Integration Testing, “big bang” is eliminated from the process
Requirements
Unit TestsBig Bang
Integration Tests
System Tests
System
Tests
• Accelerates testing
• Reduces costs
• Lowers risk
© 2014 IBM CorporationIBM Competitive Project Office 29
Performance
UAT
OAT
Tests
Rational SoftwareRational SoftwareRational SoftwareRational Software
Rational Test Virtualization Server enables incremental integration testing
Actual Component
Stubbed Component
• Units are introduced into the continuous integration cycle in a prioritized, controlled fashion.
• Units not yet built can be simulated and tested against.
© 2014 IBM CorporationIBM Competitive Project Office 30
Incremental Integration TestingIncremental Integration TestingIncremental Integration TestingIncremental Integration TestingIncremental Integration TestingIncremental Integration TestingIncremental Integration TestingIncremental Integration Testing
Rational SoftwareRational SoftwareRational SoftwareRational Software
Rational Test Workbench
• Combination of
Rational Test Workbench
Eclipse and Mobile clients
Rational Functional Tester
Rational Performance Tester
Rational Integration Tester
Rational Test Virtualization Server
• Enables sharing and reuse of test environments across the enterprise
− Sharing and reuse of virtualized components/services
− Test environments available on demand
− Enables automated build test with
Rational Test Workbench and Rational Test Virtualization Server
© 2014 IBM CorporationIBM Competitive Project Office 31
Rational Integration Tester
Rational Test Control Panel
• Virtualizes services - no coding required!
• Reuse existing RFT and RPT assets
• Integrated with RDNG, RQM, and RTC for ALM traceability
− Enables automated build test with enterprise integration testing scenarios
• Reduce test infrastructure cost and dependencies
• Leverage physical, virtual, or cloud test environments for different types of testing workloads
Rational SoftwareRational SoftwareRational SoftwareRational Software
IBM Rational Quality Management with Rational Integration Tester and Rational Test Virtualization Server The leading Agile Quality Management Solution
ManageCreate Build ReportExecute
IBM Collaborative Application Lifecycle Management
Test Management
Rational Quality Manager
Quality Dashboard
RequirementsManagement Defect
Management
© 2014 IBM CorporationIBM Competitive Project Office 32
TestVirtualization
Storage
Collaboration
Search & QueryDiscovery
Administration: Users, projects, process
Presentation:
Mashups
Best Practice Processes
Manage
Test Lab
Create
Plan
Build
Tests
Report
Results
Execute
Tests
Open Lifecycle Service Integrations
PerformanceTesting
PerformanceTesting
FunctionalTesting
UnitTesting Security and
ComplianceIntegrationTesting
Rational SoftwareRational SoftwareRational SoftwareRational Software
DEMO 2: Integration tests and test virtualization
He starts the stub in RIT and tests the access to it by
Tanuj uses Rational Integration Tester to create a stub to simulate access to a Web service
© 2014 IBM CorporationIBM Competitive Project Office 33
He deploys the stub to Rational Test Virtualization Server so it can be accessed by other users
He starts the stub in RIT and tests the access to it by interacting with the application under test
Rational SoftwareRational SoftwareRational SoftwareRational Software
Rational Quality Manager reportingDynamic reports based on Jazz Common Reports show how quality management artifacts are linked throughout the lifecycle
RQM Client: Web browser
Test Planning:
Comprehensive test plan,
Shared objectives, Scope,
Timeline, Resources, Risk
assessment
Test Execution:
Manual test execution, Use
test automation tools, Record
test results, Submit & track
defectsReporting & Dashboard:
Status & progress tracking,
Customizable live
Test Construction:
Requirement-driven testing,
Test environments
coverage, Manual test
authoring, Test lab
management
Rational DOORS Next Generation
Rational DOORS Next Generation
Collections
Requirements
Storyboards
Rational Software
Architect Design Manager
Rational Software
Architect Design Manager
Designs
Models
Rational Team Concert
Rational Team Concert
Release Plans
Iterations
Work Items
© 2014 IBM CorporationIBM Competitive Project Office 34
Jazz Team Server
RTCRDNG RQM
Core OSLC REST Services: Discovery, Query, Storage, Presentation, Data Warehousing,
Administration, Reporting, etc…
RDNG REST
Services: Rqmts
Management
RTC REST
Services: Change
Management
RQM REST
Services: Quality
Management
RSA DM
Customizable live
dashboard, Real-time
metrics and reports,
Compliance and quality
audit
Team Collaboration:
Process enactment and
enforcement, reviews, task
management, Rapid team
member on-boarding
Storyboards
Use cases
…
Diagrams
…
Work Items
Source Code
…
RSA DM REST
Services: Design
Management
Rational SoftwareRational SoftwareRational SoftwareRational Software
Rational Quality Manager reporting
Common reporting across RQM, RDNG and RTC means all QM artifacts are transparent throughout the entire lifecycle
© 2014 IBM CorporationIBM Competitive Project Office 35
Rational SoftwareRational SoftwareRational SoftwareRational Software
Rational Quality Manager – team collaborationTeam collaboration features allow QM team to easily follow best practices and make everyone more productive
RQM Client: Web browser
Rational DOORS Next Generation
Rational DOORS Next Generation
Collections
Requirements
Storyboards
Rational Software
Architect Design Manager
Rational Software
Architect Design Manager
Designs
Models
Rational Team Concert
Rational Team Concert
Release Plans
Iterations
Work Items
Test Planning:
Comprehensive test plan,
Shared objectives, Scope,
Timeline, Resources, Risk
assessment
Test Execution:
Manual test execution, Use
test automation tools, Record
test results, Submit & track
defects
Test Construction:
Requirement-driven testing,
Test environments
coverage, Manual test
authoring, Test lab
management
Reporting & Dashboard:
Status & progress tracking,
Customizable live
© 2014 IBM CorporationIBM Competitive Project Office 36
Jazz Team Server
RTCRDNG RQM
Core OSLC REST Services: Discovery, Query, Storage, Presentation, Data Warehousing,
Administration, Reporting, etc…
RDNG REST
Services: Rqmts
Management
RTC REST
Services: Change
Management
RQM REST
Services: Quality
Management
RSA DM
Storyboards
Use cases
…
Diagrams
…
Work Items
Source Code
…
RSA DM REST
Services: Design
Management
Team Collaboration:
Process enactment and
enforcement, reviews, task
management, Rapid team
member on-boarding
Customizable live
dashboard, Real-time
metrics and reports,
Compliance and quality
audit
Rational SoftwareRational SoftwareRational SoftwareRational Software
Rational CLM quality management adoption model
Tactical, opportunistic Strategic, systematic
Cost ReductionCost ReductionCost ReductionCost Reduction� Increase
Automation� Reuse Test assets� Optimize
infrastructure
Broaden ScopeBroaden ScopeBroaden ScopeBroaden Scope� Drive QM to meet
business needs� Extend QM to the
overall ALM� Consolidate QM
information
StandardizationStandardizationStandardizationStandardization� Define and track
organizational quality metrics
� Identify and promote Best Practices
Realizing StrategyRealizing StrategyRealizing StrategyRealizing Strategy� Drive Business
decisions leveraging product quality
� Use Quality as competitive differentiator
© 2014 IBM CorporationIBM Competitive Project Office 37
Cost focusCost focus Value focusValue focus
information� Facilitate
Collaboration with other stakeholders
� Manage Test Data
Practices� Promote
repeatability
differentiator� Drive continuous
quality improvements and real time decisions.
RQMRQMRQMRQM RQM + RTC + RQM + RTC + RQM + RTC + RQM + RTC + RDNGRDNGRDNGRDNG
Organizations often move through a set of phases as they adopt Quality Management
Rational SoftwareRational SoftwareRational SoftwareRational Software
Sizing up the competition
As always, I want to know what the As always, I want to know what the As always, I want to know what the As always, I want to know what the competition is up to. We are already competition is up to. We are already competition is up to. We are already competition is up to. We are already using HP Quality Center. Why should using HP Quality Center. Why should using HP Quality Center. Why should using HP Quality Center. Why should JKE choose IBM over HP Quality JKE choose IBM over HP Quality JKE choose IBM over HP Quality JKE choose IBM over HP Quality Center?Center?Center?Center?
Let me show you why IBM is Let me show you why IBM is Let me show you why IBM is Let me show you why IBM is the leader in Quality the leader in Quality the leader in Quality the leader in Quality Management spanning the Management spanning the Management spanning the Management spanning the entire application lifecycle and entire application lifecycle and entire application lifecycle and entire application lifecycle and key questions to ask the key questions to ask the key questions to ask the key questions to ask the competition.competition.competition.competition.
© 2014 IBM CorporationIBM Competitive Project Office 38
QA Manager
Rational SoftwareRational SoftwareRational SoftwareRational Software
What you should ask HP
Does HP ALM have a built-in test lab manager?
Can HP manage physical and virtual assets from within HP ALM?
© 2014 IBM CorporationIBM Competitive Project Office 39
How big is HP’s partner network for integration (i.e. how many different products can be integrated)?
Can functional tests be automated across physical and virtual machines easily?
How complete is reporting across all artifacts in the application lifecycle (i.e. can I create reports that span development artifacts, quality artifacts, and requirements?)
Rational SoftwareRational SoftwareRational SoftwareRational Software
How complete is testing in Atlassian’s solution?
• What kinds of tests can Atlassian run beyond simple JUnit?
• Does Jira Capture (Atlassian plugin, formerly called Bonfire) support test plans or test cases without requiring additional add-ons?
What you should ask Atlassian
© 2014 IBM CorporationIBM Competitive Project Office 40
ons?
• What functional and performance tests does Atlassian support?
• Are there any test lab management capabilities with Atlassian?
What types of defect creation can be automated?
• Can Atlassian’s solution create defects directly from the build engine?
• Can Atlassian automatically populate data if I create a defect in-context using other tools?
Rational SoftwareRational SoftwareRational SoftwareRational Software
Complete coverage of all quality management artifacts throughout the entire ALM
Seamless integrations with testing tools
• IBM Rational/Tivoli products
Summary
© 2014 IBM CorporationIBM Competitive Project Office 41
• IBM Rational/Tivoli products
• Rational Test Workbench and Servers (formerly Green Hat)
• 3rd party integrations via OSLC and REST
A complete Quality Management environment
• Detailed test planning
• Requirements driven testing is built-in
• Built-in manual tester
• Built-in test lab manager
• Comprehensive reporting capabilities
Rational SoftwareRational SoftwareRational SoftwareRational Software
© 2014 IBM CorporationIBM Competitive Project Office 42
© Copyright IBM Corporation 2014. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
www.ibm.com/software/rational