top100summit christina
TRANSCRIPT
![Page 1: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/1.jpg)
Automation Test Framework Continue Improvement
Welcome PresentationPresented by Christina Geng
![Page 2: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/2.jpg)
AGENDA
❶ Case Introduction❷ Testing Strategy Evolution❸ Case ROI Analysis❹ Success factor❺ Revelation
![Page 3: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/3.jpg)
Title :7 Times Evolution of Test Automation Framework under
JS Product for Efficiency Improvement
Target :
❶ Build a robust test automation framework
❷ Manage test cases efficiently
❸ Reduce test case running time and maintenance cost
❹ Shared framework under scrum development
CASE INTRODUCTION
![Page 4: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/4.jpg)
WHO AND WHAT?
Data Analysis Visualization
• reporting, online analytical processing • analytics• data mining• process mining• event processing• business performance management• benchmarking• text mining• predictive analytics
![Page 5: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/5.jpg)
WHO AND WHAT?
Data Analysis Visualization
• reporting, online analytical processing • analytics• data mining• process mining• event processing• business performance management• benchmarking• text mining• predictive analytics
Programming language: Javascript/Java
Provide service: The Visualization Service is a software module which provides various type of visualization objects that can be
used in different applications on different platforms (Web, Mobile and Desktop).
Visualization types: • A basic visual component like chart (Bar, Column Pie, Line, etc ), Geo Map, Gauge, Table or Grid.
• An advanced visual component like (Tree map, Candlestick, Tagcloud) • Above visual component with interactions and animations
• Combination of multiple visual components, it could be composed by same type of visual components or different type of component.
![Page 6: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/6.jpg)
AGENDA
❶ Case Introduction❷ Testing Strategy Evolution❸ Case ROI Analysis❹ Success factor❺ Revelation
![Page 7: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/7.jpg)
We have it!
Selenium
Data D
riven
Jenkins
Test Link
Full Automation
Developer environm
ent
What’s m
ore…2011.6 2011.12 2012.032012.06 2012.09 2012.10 2012.12
![Page 8: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/8.jpg)
VERSION-1
Test page
SilkTest
Pie Bar Line BackgroundTitle Legend Layout Tooltip Theme ValueAxis AnimationPie y y y y y yBar y y y y y y y y yLine y y y y y y y y y
Scheduler: ASTECTC Management: ExcelTC: test pageTest tool: SilkTest
![Page 9: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/9.jpg)
VERSION-1
Test page
SilkTest
+ -HTMLTest
page
1. First automation framework setup2. Overview functional test case
coverage3. Reduce testing overlap for shared
component
writing test case
maintenance
the relation between TC and record
resource
![Page 10: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/10.jpg)
We have it!
Selenium
Data D
riven
Jenkins
Test Link
Full Automation
Developer environm
ent
What’s m
ore…2011.6 2011.12 2012.032012.06 2012.09 2012.10 2012.12
![Page 11: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/11.jpg)
VERSION-2
DIV
DIVDIV
DIVTest page
Selenium
Scheduler: ASTECTC Management: Excel/XML (test manager)TC: test pageTest tool: Selenium Test Driver
Other tools• Auto generate test plan from Excel to xml• Test Manager
![Page 12: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/12.jpg)
DIV
DIVDIV
DIVTest page
VERSION-2
+ -1. Reduce test pages2. Reduce resource running cost3. Test steps turns to be keyword driven in xml4. Easy case migration5. Write test case in .xls, auto generate .xml
Selenium
DIV
DIVDIV
DIVTest page
test driver
test case sync
case unit
test data redundancy
![Page 13: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/13.jpg)
We have it!
Selenium
Data D
riven
Jenkins
Test Link
Full Automation
Developer environm
ent
What’s m
ore…2011.6 2011.12 2012.032012.06 2012.09 2012.10 2012.12
![Page 14: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/14.jpg)
VERSION-3
Wrapper API
Unified Test Driver
Abstract Test Data
Discard Test Page
And How?
![Page 15: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/15.jpg)
VERSION-3
Wrapper API
backgroudcolor
visiblelegendposition
verticalOffsethorizontalOffset
visiblelabelFont
labelColorformatString
layoutpaddingtooltip
enablecustomization
textmainValue
subValuetitle
main subSelectability
modedeSelectale
defaultSelectedItemsxAxisMemberyAxisMember
barbarMarkerSize
measureMember
initChartsByProperties([['layout.padding',[50,50,50,50]],['legend.visible',false]]);
updateChartsByProperties([['layout.padding',[50,50,50,50]],['legend.visible',false]]);
testUtil.generateChart(viz/line; TestData.a1a1m1m1)
testUtil.generateChart(viz/line; TestData.a1a1m1m1; ['legend.visible',false])
![Page 16: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/16.jpg)
VERSION-3
Abstract Test Data
publicpublic
privateprivate
testUtil.generateChart(viz/line; TestData.a1a1m1m1; ['legend.visible',false])
mySQL
testUtil.generateChart(viz/line; TestData.a1a1m1m1; legendVisibleTrue)
![Page 17: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/17.jpg)
VERSION-3
Discard Test Page
<testsetcase id="test.workflow.layout.line.showTitle" enabledtype="enabled"/><testsetcase id="test.workflow.layout.line.hideLegend" enabledtype="enabled"/><testsetcase id="test.workflow.layout.line.hideAxis" enabledtype="enabled"/><testsetcase id="test.workflow.layout.line.showAxisTitle" enabledtype="enabled"/>
titleVisibleTrue = [['title.visible',true]];titleVisibleFalse = [['title.visible',false]];titleNormal = [['title.visible',true],['title.text',"Set customer title"]];titleLong = [['title.visible',true],['title.text',testData_Text.Long]];titleAlignmentCenter = [['title.visible',true],['title.alignment',"center"]];titleAlignmentLeft = [['title.visible',true],['title.alignment',"left"]];titleAlignmentRight = [['title.visible',true],['title.alignment',"right"]];legendVisibleTrue = [['legend.visible',true]];legendVisibleFalse = [['legend.visible',false]];legendHierarchicalTrue = [['legend.visible',true], ['legend.isHierarchical', true]];legendPositionLeft = [['legend.visible',true], ['legend.position',"left"]];legendPositionRight = [['legend.visible',true], ['legend.position',"right"]];legendPositionTop = [['legend.visible',true], ['legend.position',"top"]];legendPositionBottom = [['legend.visible',true], ['legend.position',"bottom"]];
![Page 18: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/18.jpg)
VERSION-3
Unified Test Driver
selenium1
selenium2{StartBrowser}IE, Chrome, Firefox (selenium2)
RemoteWebDriver webDriver = null;
webDriver = new FirefoxDriver();
Robot()
ImageComparer
![Page 19: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/19.jpg)
VERSION-3
Unified Test Driver
selenium1
selenium2
{StartBrowser}Safari (selenium1)
SeleniumServer seleniumServer = new SeleniumServer();
seleniumServer.boot();
selenium = new DefaultSelenium("localhost", 4444, "*safariproxy "+ safariLocation,urlRoot);
CommandExecutor executor = new SeleneseCommandExecutor(selenium);
DesiredCapabilities dc = new DesiredCapabilities();
webDriver = new RemoteWebDriver(executor, dc);Robot()
ImageComparer
![Page 20: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/20.jpg)
VERSION-3
Unified Test Driver
selenium1
selenium2 {TakeSnapshot}Point p = this.getElementOffset(targetElement);
Dimension d = this.getElementSize(targetElement);
BufferedImage bi = new Robot().createScreenCapture(p.x, p.y, d.width, d.height);
Robot()
ImageComparer
![Page 21: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/21.jpg)
VERSION-3
Unified Test Driver
selenium1
selenium2
ImageComparer
{FileCompare}cmd.exe ImageComparer.exe –file filepath1 filepath2 – threshhold 100
Robot()
![Page 22: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/22.jpg)
We have it!
Selenium
Data D
riven
Headless
Test Link
Full Automation
Developer environm
ent
What’s m
ore…2011.6 2011.12 2012.03 2012.06 2012.09 2012.10 2012.12
![Page 23: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/23.jpg)
VERSION-4
Tooltip Selection
Gesture
CSSButton
Link
Dropdown
Checkbox
Table Text
Scroll
Hover
Animation
Theme
Upload
Download
Tooltip Selection
Gesture
![Page 24: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/24.jpg)
VERSION-4
Selection
![Page 25: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/25.jpg)
VERSION-4
Gesture
![Page 26: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/26.jpg)
VERSION-4
var eventObj = {type : originalEvent.type,timeStamp : originalEvent.timeStamp,pageX : originalEvent.pageX,pageY : originalEvent.pageY
};eventRecord.push(eventObj);
var handler = function(event){var clickEvt = document.createEvent('MouseEvents');clickEvt.initMouseEvent(event.type, event.pageX,
event.pageY…);d3.select('.main .plot')[0]
[0].dispatchEvent(clickEvt)};testAPI_lasso_heatmap = [
{"type":"mouseover","timeStamp":1348739963751,"pageX":126,"pageY":94},{"type":"mouseover","timeStamp":1348739963751,"pageX":485,"pageY":94},{"type":"mousemove","timeStamp":1348739963766,"pageX":485,"pageY":208},{"type":"mousemove","timeStamp":1348739963773,"pageX":126,"pageY":208}
]
![Page 27: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/27.jpg)
VERSION-4
Record
Replay
![Page 28: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/28.jpg)
VERSION-4
start to checkdiff image folderdiff image location
p4 folder
replace BM directly
![Page 29: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/29.jpg)
We have it!
Selenium
Data D
riven
Headless
Test Link, Jenkins
Full Automation
Developer environm
ent
What’s m
ore…2011.6 2011.12 2012.03 2012.06 2012.09 2012.10 2012.12
![Page 30: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/30.jpg)
VERSION-5
How to arrange test case?
![Page 31: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/31.jpg)
VERSION-5
Test Link
TC (.xls/.xml)
Executable TP
import
export
Result logsave
![Page 32: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/32.jpg)
VERSION-5
Continue Integration
![Page 33: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/33.jpg)
1. Trigger
CI Server Test Executor
Build Machine Code repository
Manual Auto
TC mngmt
Test Case Pool
Test Driver
Selenium2 Java/C++
Web Server
2.TP/Platform/build info
resources
4. get assets
libs
4. get test plan3. Build info 5. result write back
5. result log
.xml.xls
VERSION-5
Where we are.
![Page 34: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/34.jpg)
1. Trigger
CI Server Test Executor
Build Machine Code repository
Manual Auto
TC mngmt
Test Case Pool
Test Driver
Selenium2 Java/C++
Web Server
2.TP/Platform/build info
resources
4. get assets
libs
4. get test plan3. Build info 5. result write back
5. result log
.xml.xls
VERSION-5
Test Link
JenkinsJava Tool
PerforceWhere we are.
![Page 35: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/35.jpg)
We have it!
Selenium
Data D
riven
Headless
Test Link, Jenkins
Full Automation
Developer environm
ent
What’s m
ore…2011.6 2011.12 2012.03 2012.06 2012.09 2012.10 2012.12
![Page 36: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/36.jpg)
Engineering
![Page 37: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/37.jpg)
VERSION-6
http://casperjs.org/
![Page 38: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/38.jpg)
VERSION-6
Paint
Layout Display
http://phantomjs.orgPhantomJS (Headless Webkit)
![Page 39: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/39.jpg)
VERSION-6
Test driver inner logic
CasperJS IE/Safari/FF/Chrome
configBrowser?
Open browser
RunJS (test step)
….
Take Snapshot
Result validation
Save result log
Read JS listTP
PhantomJS:1. open webkit2. runJS3. Take snapshot4. close
not completenot complete
Reporting
![Page 40: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/40.jpg)
And that’s OK?
![Page 41: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/41.jpg)
We have it!
Selenium
Data D
riven
Headless
Test Link, Jenkins
Full Automation
Developer environm
ent
What’s m
ore…2011.6 2011.12 2012.03 2012.06 2012.09 2012.10 2012.12
![Page 42: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/42.jpg)
trigger
CI Server Test Executor
Build MachineP4 server
TC Management
Test Case Pool
Test Driver
Selenium2 Java/C++
get resources
sync up
.java
per check-in
local
Test Executor
result log
VERSION-7
Casper/PhantomJS
![Page 43: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/43.jpg)
trigger
CI Server Test Executor
Build MachineP4 server
TC Management
Test Case Pool
Test Driver
Selenium2 Java/C++
get resources
sync up
.java
per check-in
local
Test Executor
result log
VERSION-7
Casper/PhantomJS
Eclipse Plug-in
PerforceJenkins
![Page 44: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/44.jpg)
1. Set up environment
2. Test case execution
3. Test result reporting
4. Maintenance
CASE ROI ANALYSIS
1) Test machine2) Assets (util, BM…) 3) TP4) Libs
1) Browser compatibility2) Run in headless
1) Email sending2) Sonar integrated with CI
1) BM update2) TC management
Functional Test: BAT (nightly), FA
1000+ cases
4.5h47min
![Page 45: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/45.jpg)
SUCCESS FACTOR ❶ ❷❸❹
Focus
![Page 46: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/46.jpg)
SUCCESS FACTOR ❶ ❷❸❹
Team work
![Page 47: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/47.jpg)
SUCCESS FACTOR ❶ ❷❸❹
Continue Improvement
![Page 48: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/48.jpg)
SUCCESS FACTOR ❶ ❷❸❹
Learn 360°
![Page 49: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/49.jpg)
OUR REVELATION
不被传统的测试方法束缚;从产品用到的技术找到突破;持续改进的思想;不断尝试的理念
![Page 50: Top100summit christina](https://reader033.vdocument.in/reader033/viewer/2022050907/55b93d58bb61eba57d8b467d/html5/thumbnails/50.jpg)
CONTRIBUTORXiaolu, Ye
Christina, GengHelen, JiangChen, ZhouPeipei, Jiang
Coco, DaiGrace, ZhaoMinghao, LuKent, Wang Izzie, Wang
Phil, Xu
SUPPORTED FROMDaoyu, Wei
Li, ZhaoJimmy, YangAlex, FengAlex, Su Jiwei, Li ... …
THE ENDNO