![Page 1: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/1.jpg)
Automating Qt GUI Tests10 Pitfalls And How To Avoid Them
Qt Developer Days 2012
by Reginald Stadlbauer
![Page 2: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/2.jpg)
About me
Name: Reginald Stadlbauer Company: froglogic GmbH, vendor of Squish for Qt and Squish Coco Position: co-founder and CEO Worked as Software Engineer at Trolltech and the KDE project
![Page 3: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/3.jpg)
Overview
Types of Testing Why Automate? Pitfalls 1-10 Demo based on “Rohde & Schwarz PowerViewerPlus”
![Page 4: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/4.jpg)
Types of Testing
Unit Testing Performance Testing ... Functional GUI Testing
- Black/Gray Box Testing- Assume user's point of view- Automate to spot regressions- Combinable with profiling, coverage and other analysis and monitoring tools
![Page 5: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/5.jpg)
Why Automate?
Faster- Get results quicker- Run more tests in the same time
Trivial to replay in different configurations Reliable, reproducible and repeatable Relieve testers from monotonous tasks
![Page 6: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/6.jpg)
But...
Automating GUI tests is not trivial Following best practices is vital for the success of automated GUI tests
![Page 7: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/7.jpg)
1. Rely on capture and replay
Produces massive test scripts Not readable Not maintainable No code re-use possible Brittle against changes in the UI
Solution: Scripting & Refactoring
![Page 8: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/8.jpg)
2. Use primitive macro language
Limited to small set of features No way to “break out” No way to utilize 3rd party libraries (database access, etc.) No way to deal with dynamic tests
Solution: Use scripting solution for test automation
![Page 9: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/9.jpg)
Scripted Approach vs. Capture & Replay
![Page 10: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/10.jpg)
3. Rely on screen coordinates
Addresses screen positions and not UI controls Breaks with UI layout changes Depends on GUI style and platform Scripts hard to understand
Solution: Address objects based on properties
![Page 11: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/11.jpg)
4. Rely on screen captures / OCR
No knowledge of GUI controls Too much heuristics Depends on irrelevant data (colors, fonts, etc.) Many incorrect fails / errors
Solution: Identify on and compare object properties
![Page 12: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/12.jpg)
5. Rely on “Windows” or “Accessibility” test tools
Only “knows” standard Windows controls Cannot drill into Qt / QML / Quick / WebKit controls Object identification based on limited amount of properties Not cross-platform
Solution: Use a tool which understands Qt controls
![Page 13: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/13.jpg)
Example: Widget Recognition Options
Very BAD:
MouseClick(132, 367)
BAD:
MouseClick('Tree', 30, 136)
BAD:
MouseClick(
FindObjByImg('item-image.png'))
GOOD:
ClickItem('Tree', 'Event')
![Page 14: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/14.jpg)
6. Tests embedded in application
Tempting to test API rather than GUI Application crash or freeze not handled well Can only test one application per test case Not suitable for remote testing (embedded devices, mobile) Modifies application
Solution: Run test in a separate process
![Page 15: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/15.jpg)
7. Rely on unique Qt objecName
Burden for developers Not realistically doable if testing is introduced later Need uniqueness checking
Solution: Use multi-property naming
![Page 16: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/16.jpg)
8. Rely on AUT's object hierarchy
Long and unreadable names Relies on application internal “helper widgets” Small layout changes breaks naming
Solution: Use multi-property naming
![Page 17: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/17.jpg)
9. Create tests “on the side”
Development resources are already restricted There is always “one more important dev task” Easy to delay “until tomorrow”
Solution: Dedicated resource for testing
![Page 18: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/18.jpg)
10. Setup automation “when ready”
Nobody runs the tests and sees the fails/errors Tests will become unmaintained and not work anymore Tests will be forgotten
Solution: First task: set up automation, then start creating tests
![Page 19: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/19.jpg)
11. There are more...
Thinking there are only 10 pitfalls :-)
![Page 20: Automating Qt GUI Tests · 2018-11-19 · Automating Qt GUI Tests 10 Pitfalls And How To Avoid Them Qt Developer Days 2012 by Reginald Stadlbauer. ... Types of Testing Unit Testing](https://reader033.vdocument.in/reader033/viewer/2022053014/5f11f798e1bc3713f204209f/html5/thumbnails/20.jpg)
Squish for Qt Demo on Rohde & Schwarz PVP
Discuss Record & Replay Verifications Object naming Refactoring & Scripting Screenshot Verifications Keyword driven testing
Q & A