designing hardware-independent testing laboratory api · designing hardware-independent estingt...
TRANSCRIPT
![Page 1: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/1.jpg)
Designing Hardware-independent Testing Laboratory API
Paweª Wieczorek
August 22, 2019
Samsung R&D Institute Poland
![Page 2: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/2.jpg)
Outline
Motivation
Testing laboratory layers
Case study
Summary
1/17
![Page 3: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/3.jpg)
Motivation
![Page 4: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/4.jpg)
Use cases
Automated testing
New
software
Allocate
resources
Perform
tests
Release
resources
Return
results
Direct access (hacking)
Issue
investigation
DUT
acquisitionInteraction
DUT
releaseIssue solved
2/17
![Page 5: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/5.jpg)
Abstractions (https://elinux.org/Test_Standards)
TM Test Manager: Actions initiation
TSTest Scheduler: Resource
allocation, scheduling
DUT-C
DUT Control: Controlling power,
providing network, ensuring
communication, grabbing logs
DUT Device Under Test
3/17
![Page 6: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/6.jpg)
Common laboratory structure
DUT
Control
Test
scheduler
Test
manager
4/17
![Page 7: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/7.jpg)
Decoupled laboratory structure
Test
manager
Test
scheduler
DUT
Control
DUT
5/17
![Page 8: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/8.jpg)
Devices
6/17
![Page 12: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/12.jpg)
Testing laboratory layers
![Page 13: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/13.jpg)
Challenges
Knowledge
Which actions are necessary? Where can it be performed? How to do it?
Responsibilities
Who performs given action?
Sharing
Who can use DUT? How can DUT be used?
10/17
![Page 14: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/14.jpg)
Implementation
Test Manager (minimal)
� initiate actions
� list (or cancel) currently performed actions
Test scheduler (generic)
� list available resources, request speci�c ones
� acquire assigned resources (then prolong, �nally release)
DUT Control (tricky)
� boot (and login)
� execute commands
� copy �les 11/17
![Page 15: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/15.jpg)
Case study
![Page 16: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/16.jpg)
Test Manager
Strengths
� Requires only preparing test plan
� Test plans can be reused among various projects
Weaknesses
� Keeping compliance
� Catching up with others (e.g. LAVA, SQUAD)
12/17
![Page 17: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/17.jpg)
Test Scheduler
Strengths
� Users treated equally
� Resource type-agnostic
Weaknesses
� Requires additional agent
� Capabilities declared up front
13/17
![Page 18: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/18.jpg)
DUT Control
Strengths
� Only some knowledge required
� Uni�cation possibility
Weaknesses
� Hard initial setup
� Often unique for a given testing laboratory
14/17
![Page 19: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/19.jpg)
Summary
![Page 20: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/20.jpg)
Outcome
� Unable to demo without speci�c hardware
� Risky large scale deployments
� Responsibilities division allows easier onboarding
15/17
![Page 21: Designing Hardware-independent Testing Laboratory API · Designing Hardware-independent estingT Laboratory API Paweª Wieczorek August 22, 2019 Samsung R&D Institute Poland. Outline](https://reader034.vdocument.in/reader034/viewer/2022050503/5f9586af97a0f54ae919e7ea/html5/thumbnails/21.jpg)
Conclusion
� User-centric approach resulted in smaller building blocks
� Smaller blocks could be easier swapped or used independently
� Improvement needs more reuse instead of rewrite
https://github.com/SamsungSLAV
16/17