kubernetes your tests! automation with docker on google cloud platform
TRANSCRIPT
By Arik Lerner & Waseem Hamshawi
Kubernetes your Tests! Testing with Docker on Google Cloud Platform
The JourneyBased On True Story!
{
“Name”: “Arik Lerner”,
“Role”: “Automation Infra Team Leader at LivePerson”,
“Data”: [
“3 years as Team Leader in Liveperson” ,
“2 years Java developer Reporting Platform Liveperson”
],
“Hobbies”: “Private pilot”
}
Active Prevention Department
{
“Name”: “Waseem Hamshawi”,
“Role”: “Senior Automation Infra Developer at Liveperson”,
“Data”: [
“2.5 years at Liveperson”,
“github: waseemh.github.io”,
],
“Hobbies”: “Running”
}
Waseem is Here!
Active Prevention Department
Short Story
From Manual Post Deployment Verification
TOAutomation
❏ Fast feedback
❏ Stable Environment
❏ Isolation
❏ Investigation tools - Hard to support (Video/HAR)
❏ Scale
❏ Efficiency
❏ Browser diversity
❏ Mobile Support
Requirements
Selenium HubRunning Test
Static VMs
Supported Requirements
Browser diversity
Mobile Support
Parallelism
Traditional Selenium Grid
Ranking:
Tooling
Deployment/Configuration
Feedback
Isolation
Efficiency
Scalability
❏ Less stable❏ Crashes - require restarts❏ Cancel build in progress leave open sessions
❏ Investigation tools - Hard to support (Video/HAR)
❏ Manual grid node Configuration/deployment
❏ Not scalable
❏ Slow feedback
❏ Not efficient - idle grid machines
Facing Issues
Cattle vs. Pets
VM
Container
● When they get ill you nurse them back to health
● You name them! And take of them!
● When they get ill you get another one!
● They are almost identical to one another
● Don't have specific names. Given numbers
● Can be simply replaced if they die
What Docker gives us ?
● Build it once run it everywhere!
● Resource Isolation
● Test Isolation
● Package format
● Implementation of a container
Docker
Chrome FirefoxVNC Server
Selenium Server Hub/Node JAR
Mobile Android
What is in a Selenium Container ?
VM
VM
VM
VM
NodesHUB
N Hub’s
Running Tests
Selenium Grid Architecture Makeover
DEMO
We need more than just Packaging and Isolation:
● Orchestration - Distribute containers
● Health Checks
● Resilience and Self-healing
● Scaling - making jobs bigger or smaller
Kubernetes Orchestrator
VM
VM
VM
VM
PodsHUB
N Hub’s
Running Tests
Selenium Grid Architecture Makeover
Testing on the Cloud!
Benefits of Cloud Platform
● Kubernetes cluster in a click of a button (GCP)
● No idle machines - Use when needed
● Costs per build/test - measurable
● Not part of Liveperson network
● Geolocation simulation
DEMO
Build Life Cycle
GCP - Selenium Grid lifecycle
Run tests
Kill Selenium Grid
Scale Down GCP Instances
Scale Out GCP Instances
Create Selenium Grid container cluster
Requirements
Requirements
❏ Fast feedback
❏ Stable Environment
❏ Isolation
❏ Investigation tools - Hard to support (Video/HAR)
❏ Scale
❏ Efficiency
❏ Browser diversity
❏ Mobile Support
Ranking:
Tooling
Deployment/Configuration
Feedback
Isolation
Efficiency
Scalability
❏ Investigation tools - Video tools not reliable and not stable.
❏ Feedback ❏ Improved BUT still needs to wait for nodes to
connect into Hub❏ Point of failures
❏ Hub/Nodes is more failure points in the architecture that not really necessary
Facing Issues
waseemh-mac$ java -jar selenium-server-standalone-2.53.1.jar -role
Selenium server can run in one of the following roles: hub as a hub of a Selenium grid node as a node of a Selenium grid standalone as a standalone server not being a part of a grid
Test Hub Selenium Server(Node)
Selenium Standalone
Non-Grid Solution● Fast → Deploy Selenium Standalone containers on-demand● Short-lived → Selenium containers are destroyed once test ends● Distribute and manage containers with Kubernetes
TestcreateRemoteWebDriverSession(caps)
VM
VM
VM
VM
new RemoteWebDriver(External Endpoint)
Test Cluster
?
Ephemerals API
Deploy
Endpoint IP:Portnew RemoteWebDriver(Endpoint IP:Port)
createRemoteWebDriver(desiredCapabilities)
TestTestTestTestTestTestTestTestTest
JUnit Integration@Rulepublic EphemeralResource<RemoteWebDriver> seleniumResource =
new EphemeralResource( new SeleniumEphemeral.Builder(deploymentContext) .withBrowser(FIREFOX_42) .withVideoRecording(true) .build());
@Testpublic void test() { RemoteWebDriver remoteWebDriver = seleniumResource.get(); .....}
DEMO
Challenges● Appium crashes/unresponsive, adb server restarts
● Android SDKs environment installation/maintenance
● Appium↔WebDriver API Versions Incompatibility
● Difficult to parallelise mobile tests● Scale mobile tests ?!
Mobile Testing
Emulator
Ephemeral’s Way
Emulator
Appium WebDriver
Test
Install (APK URL)
Ephemerals API
createMobileDriver(capabilities) Deploy
Appium Endpoint IP:Port
new AndroidDriver(Appium IP:Port)
DEMO
Requirements
Ranking:
Tooling
Deployment/Configuration
Feedback
Isolation
Efficiency
Scalability
Requirements
❏ Fast feedback
❏ Stable Environment
❏ Isolation
❏ Investigation tools - Hard to support (Video/HAR)
❏ Scale
❏ Efficiency
❏ Browser diversity
❏ Mobile Support
Beyond Selenium...
Deploy any test dependency inside a container
● Web Servers (/modules/nginx)● Temporary Databases (/modules/mysql)● Mock Servers (/modules/wiremock)● Any Docker Image (/modules/custom)
Talking Together...
Web Application
Nginx Ephemeral Selenium Ephemeral
EP
Test Script
EP
WireMock Ephemeral
EP
RESTAPI
HTTP
Ephemerals - Future Plans
● REST API (for non-Java tests)
● Support more cluster management systems:
○ Apache Mesos
○ Docker Swarm
● More modules
Takeaways
● Enhance traditional Selenium tools
● Containerization - Short lived, throwaway instances
● Scalable tests with Containers Orchestration
● Benefits of Testing on Cloud Platforms
THANK YOU!