deepak bhimaraju; edward allen test challenges in the cloud
TRANSCRIPT
![Page 1: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/1.jpg)
DEEPAK BHIMARAJU; EDWARD ALLEN
TEST CHALLENGES IN THE CLOUD
![Page 2: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/2.jpg)
Speakers
Edward Allen18 years QA experienceQA Director - Oracle Service CloudCo-author of a chapter in the book “Experiences of Test Automation”
![Page 3: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/3.jpg)
Speakers
Deepak BhimarajuWorked as a QA Engineer at Oracle IndiaRecently moved to a development role
![Page 4: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/4.jpg)
The Cloud – Our Partly Sunny Version
Multi-tenant
Multi-version
Enterprise
![Page 5: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/5.jpg)
The Cloud – Our Partly Sunny Version• Email• Chat• Collaboration• Case Management
Software as a Service
• Custom Data• Application Development• Middleware• Managed Frameworks
Platform as a Service
• Coming Soon?Infrastructure as a Service
Consume
Build
Host
![Page 6: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/6.jpg)
On Premise Deployment: BenefitsKnown hardwareTried and Tested ConfigurationsPhysical proximity
Image Credits:SilverStartalk, http://en.wikipedia.org/wiki/Computer_network_diagram#/media/File:Sample-network-diagram.png
![Page 7: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/7.jpg)
On Premise Deployment: BenefitsLegacy software providing stability over timeTest approach or process is a known quantity
Primary test focus is functionalCompatibility, Capacity, Scalability, Security testing
Image Credits:SilverStartalk, http://en.wikipedia.org/wiki/Computer_network_diagram#/media/File:Sample-network-diagram.png
![Page 8: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/8.jpg)
On Premise Deployment: Challenges
ScalabilityMaintenanceSetup CostsLatency*
Image Credits:Erik Pitti, https://www.flickr.com/photos/epitti/2370848133/
![Page 9: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/9.jpg)
Promises of The Cloud
ScalableMaintainableLow Latency*Secure
Image Credits:Christi Gain, https://www.flickr.com/photos/christigain/5636888777/
![Page 10: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/10.jpg)
Challenges of The Cloud
Reliance on the Cloud Operations teamPinning responsibility can take time (environment or code or both)Finding bugs early as fixing takes more time.
![Page 11: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/11.jpg)
Test Challenges: Multi-Tenant Saas
SecurityTransaction Volumes
Shared servicesSpillover from other tenants
Scale and CapacityTenant growthData volume per tenant
![Page 12: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/12.jpg)
Test Challenges: Multi-Version SaaS
Rapid Scheduled Deliveries ZERO down-timeNo customer patience for break-fixes or post-upgrade work (promise of the Cloud)
Customers choose when to uptake featuresSingle-version Shared Services
![Page 13: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/13.jpg)
Test Challenges: PaaS
Managed Frameworks
App Development
Custom DataReporting
Workflow and Business Logic
![Page 14: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/14.jpg)
Test Challenges: IaaS
Functional Testing is minimalMust be reliable, secure, compatible
Image Credits:Torkild Retvedt, https://www.flickr.com/photos/torkildr/3462607995/
![Page 15: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/15.jpg)
Test Challenges: Architecture
Reference: Li, et al. (2011). Characteristics of multiple component defects and architectural hotspots: A large system case study. Empirical Software Engineering, 16 (5), 667 702. ‐
• Multiple Component Defects (MCDs): Architectural defects which typically span more than one component encompassing multiple, interacting components.
• Cost of a single MCD can be amplified by the number of tenants.
![Page 16: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/16.jpg)
• Cost of MCDs is as much as 20 times that of a Functional Defect
Test Challenges: Architecture
92%
8%
52%
48%Regular Defect
MCD
Reference: Li, et al. (2011). Characteristics of multiple component defects and architectural hotspots: A large system case study. Empirical Software Engineering, 16 (5), 667 702. ‐
% of Total App Defects
% of Total Repair Efforts
![Page 17: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/17.jpg)
Solutions: General
Promote and Value Testing as a ProfessionGreater decision making authority
Technical LevelProduct Level
Popularize in UniversitiesFind new test perspectives and approachesFinding the weakest links in the cloud
![Page 18: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/18.jpg)
Solutions: General
Utilize the “inter-regnum”Work for other teams on the cloudCreate a common Knowledge BaseOffer incentives to move developers to QAEqual Pay
![Page 19: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/19.jpg)
Solutions: Functional TestingRapid Release
Automate quickly and in parallelManual scripted tests minimizedExploratory testing
![Page 20: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/20.jpg)
Solutions: Cloud Testing Gap
Functional Analysis (Integration and System Test)
Structural Analysis (Reliability, Security, Scalability, Capacity,
Compatibility)
Static Analysis Unit Test
Tester
Developer
Attribution to Dr. Bill CurtisSVP and Chief Scientist, CAST Research Labs
![Page 21: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/21.jpg)
Solutions: Structural Testing
Testing cannot be “after the fact”Much of the work isn’t submitting bug reportsTesters have greater product expertise than developers
![Page 22: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/22.jpg)
Solutions: Structural Testing
Testers lead on capacity, scalability, performance, securityFinding best solution among many good options
Or least worse among many bad optionsContinual, rapid, iterative development
![Page 23: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/23.jpg)
Solutions: Structural Testing
Need to be experts in Customer usageAvailabilitySystem administrationCloud app configurationSecurityScalabilityCapacityPerformanceNetworkingAPIs
![Page 24: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/24.jpg)
Solutions: Structural TestingLearn craft and discipline of teaching and leading
Testers must leadTeach usage patterns, data volume modeling, exploitsUnique position to drive discussion
Testers train developersPrevent and RemoveFewer defects created in the first place
Secure codingApp configurationPerformance
![Page 25: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/25.jpg)
Solutions: Structural Testing
Expertise demanded of Cloud Testing is ratcheted upWe must be up to the challenge
Cannot be satisfied to test boundary conditions and alpha-numeric fieldsCannot continue the exploratory/traditional manual testing debate
![Page 26: DEEPAK BHIMARAJU; EDWARD ALLEN TEST CHALLENGES IN THE CLOUD](https://reader036.vdocument.in/reader036/viewer/2022062804/56649e115503460f94afdd93/html5/thumbnails/26.jpg)
DEEPAK BHIMARAJU; EDWARD ALLEN
TEST CHALLENGES IN THE CLOUD
SPECIAL THANKS:
BRAD PASCOE, Oracle Corp.
PRASAD NAGARAJU, Oracle India
NILOY GUPTA, CMU, Pittsburgh