testing strategies for asymmetric environments · use techniques such as testing stubs, using...
TRANSCRIPT
![Page 1: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/1.jpg)
V1.0 | 2019-05-29
Jeffrey Fortin, Product Manager, Code Testing
Testing Strategies for Asymmetric Environments
![Page 2: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/2.jpg)
2
1. Asymetric Environments
2. Testing in an Asymetric Environment
3. Summary
4. Questions
Agenda
![Page 3: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/3.jpg)
3
Software Based Systems are Evolving RapidlyAsymetric Environments
• Hundreds of files• Thousands of Lines of Code• Lots of “connections”• Lots of safety concerns
SOFTWAREEMBEDDED SOFTWARE
+SIZE
ENTERPRISESOFTWARE
+Connections
• Thousands of files• Millions of Lines of Code• Few “connections”• Few safety concerns
+Safety
![Page 4: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/4.jpg)
4
Today’s challenge: Meet quality objectives while controlling costAsymetric Environments
![Page 5: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/5.jpg)
5
Today’s challenge: Meet quality objectives while controlling costAsymetric Environments
Code size expandingSame regulatory requirementsSame or higher quality expectationsSame budgets and schedules
A strategy is needed …
![Page 6: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/6.jpg)
6
Environments and DevOpsAsymetric Environments
Development• New Code• Unit Tests• Static Analysis• Code Coverage criteria met
• Development Host
Integration
• Latest code base• Updated continuously
QA• Integration Tests• Regression Tests• Updated Usually Overnight
Staging
• Simulate the production environment
• Perform tests that can’t be done sooner
Production• End user is now your tester
• Bug reports• External monitoring
![Page 7: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/7.jpg)
7
1. Asymetric Environments
2. Testing in an Asymetric Environment
3. Summary
4. Questions
Agenda
![Page 8: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/8.jpg)
8
Multiple ConfigurationsTesting in an Asymetric Environment
Development• Windows Host• Microsoft Visual Studio
Integration• Linux Build Server• GCC Compiler
QA• Docker• Simulator• Cross Compiler
Staging• Sim/Stim• HIL• Signal Generators• Network Simulation• Reference Hardware
Production• Device Management
• Embedded “Target”
![Page 9: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/9.jpg)
9
Typical Embedded Code ExampleTesting in an Asymetric Environment
typedef unsigned long uint32_t;/* Memory Mapped Hardware Registers with *special* properties */uint32_t volatile REG1;static inline uint32_t getREG1();
/******************************************************************************************//* take action based on the value in REG1 and REG2 */int avoid_collision() {
if ((REG1 & 0xFF00) & 0xFE00) {return 1;
}return 0;
}
Can the code logic be tested early?
![Page 10: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/10.jpg)
10
Embedded Code Example Refactored for TestingTesting in an Asymetric Environment
/******************************************************************************************//* Better implementation that allows for stubbing of the special registers */inline uint32_t getREG1() {
/* Using *inline* should give same performance as macros or inline register access */return REG1;
}int avoid_collision_api() {
/* Notice calls through stub-able accessor functions */if ((getREG1() & 0xFF00) & 0xFE00) {
return 1;}return 0;
}
typedef unsigned long uint32_t;/* Memory Mapped Hardware Registers with *special* properties */uint32_t volatile REG1;static inline uint32_t getREG1();
![Page 11: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/11.jpg)
11
Simulating Hardware StatesTesting in an Asymetric Environment
Temperature is a 2’s compliment binary
Using a stub routine, we can easily test for any temperature setting
We can also test for “impossible” bit combinations
![Page 12: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/12.jpg)
12
Injecting FaultsTesting in an Asymetric Environment
Bits 26-30 represent fault conditions
Testing with stubs we can stimulate the code to respond to the fault
We can even set multiple fault conditions
![Page 13: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/13.jpg)
13
Call to getHW_TEMP_STATUS()
Testing ExampleTesting in an Asymetric Environment
TestDriver
ModuleUnder
Test
DummyFunctions
(stubs)
RealFunctions
![Page 14: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/14.jpg)
14
Using Test Automation in the Development EnvironmentTesting in an Asymetric Environment
Development Environment is resource richThe automation system can cycle through
hundreds of test parameters and re-run tests on any code that has changed
Tests can be setup to run on command by anyone
![Page 15: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/15.jpg)
15
Collecting Testing Metrics in the Development EnvironmentTesting in an Asymetric Environment
Get early indication of quality risksTesting coverage is a key metricOther metrics help to make better decisions
![Page 16: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/16.jpg)
16
1. Asymetric Environments
2. Testing in an Asymetric Environment
3. Summary
4. Questions
Agenda
![Page 17: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/17.jpg)
17
Leverage the Development Environment as much as possible
Use techniques such as testing stubs, using multiple compiler configurations and test automation
By doing so, embedded applications can be developed more efficiently and at lower risk than by only testing in the final Production Environment.
ConclusionSummary
![Page 18: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/18.jpg)
18
1. Asymetric Environments
2. Testing in an Asymetric Environment
3. Summary
4. Questions
Agenda
![Page 19: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/19.jpg)
19
Testing Strategies for Asymmetric EnvironmentsQuestions
!?
![Page 20: Testing Strategies for Asymmetric Environments · Use techniques such as testing stubs, using multiple compiler configurations and test automation By doing so, embedded applications](https://reader034.vdocument.in/reader034/viewer/2022042221/5ec8037143fef409d772d555/html5/thumbnails/20.jpg)
20 © 2019. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. V1.0 | 2019-05-29
Author:Fortin, JeffreyVector
For more information about Vectorand our products please visit
www.vector.com