embedded tdd cycle - deliver tdd cycle... · 2011. 9. 10. · development environment acceptance...
Post on 22-Jan-2021
1 Views
Preview:
TRANSCRIPT
1
Embedded TDD Cycle –The First 3 Years
Markku Åhman and Timo PunkkaSchneider Electric
first.last at schneider-electric.com
Agenda for today:
• Embedded Characteristics
• Unit Testing
• Dual Targeting
• Acceptance Testing Using Simulation
• Testing in Target HW
• Results and Summary
2
Embedded characteristicsCross-compilationEvent-drivenRTOS dependenciesReal time constraintsLow level HW dependenciesLimited tools supportRobustness
7
+2Low resource budgetLimited user interface
150KLOC CLegacy Code from 10+yrs RTOS /w taskingModular multi-uCSafety critical (property andhuman)
Main BoardRenesas uC2M Flash2M RAM20MHz
3
4
Problem statement:
“We can’t automate the testing of this”
Our Cycles
Unit Tests indevelopmentenvironment
Acceptance testsusing simulation
Automated testsIn target HW
Explorative Testing
Adapted from James Grenning, Embedded Test Driven D evelopment Cycle, 2004
Lenght of feedback cycle LongShort
5
Unit Test Build
#include <unity.h>
#include "test/mock/code/loopdb/Mockaddr_alarm_interface.h"
#include "code/utils/lists.h"#include <something.h>
Test build is described in test file using keywords
6
Has worked:CMock (be careful)Small group workshopsHigher level tests (through pinch point)Coaches as visitorsWebinarsRunning tests on PC
Dual Targeting
DrawTextLine();
Main();
IndicateAlarm();
DrawAlarmDisplay();
DrawCharacter();
DrawPixel();
display memory
RealImplementation
?DrawTextLine();
Main();
IndicateAlarm();
DrawAlarmDisplay();
DrawCharacter();
DrawPixel();
display memory
DrawTextLine();
Windows GUIComponent
SimulationImplementation
7
Dual Targeting
DrawTextLine();
Main();
IndicateAlarm();
DrawAlarmDisplay();
DrawCharacter();
DrawPixel();
fakedisplay memory
DrawBitmap();
Windows GUIComponent
UpdateDisplay();
ConvertToBitmap();
SendTextLineTo TestDriver();
Dual Targeting
8
Benefits of Simulation
• Running in development environment
• Fast build-run cycles
• Better debugging environment
• More portable code
Challenges in simulation
• The real-time
• Multi-tasking (OS)
• Distributed systems
9
Timing multiple executables
Executable A
for (;;) {Run_A();Win32.Sleep(1 ms);
}
Executable B
for (;;) {Run_B();Win32.Sleep(5 ms);
}
Ideal execution
Run A, t=0 Run B, t=0
Run A, t=1
Run A, t=2
Run A, t=3
Run A, t=4
Run B, t=5Run A, t=5
Run A, t=6
Executable A Executable Brealtime
0 ms
1 ms
2 ms
3 ms
4 ms
5 ms
6 ms
10
Possible execution
Run A, t=0 Run B, t=0
Run A, t=1 Run B, t=5
Run B, t=10
Executable A Executable Brealtime
0 ms
10 ms
20 ms
30 ms
40 ms
50 ms
60 ms
Run A, t=2
Run A, t=3
Run A, t=4
Run A, t=5
Run A, t=6
Run B, t=15
Run B, t=20
Run B, t=25
Run B, t=30
Another possible execution
Run A, t=0 Run B, t=0
Run A, t=1
Run B, t=5
Run B, t=10
Executable A Executable Brealtime
0 ms
10 ms
20 ms
30 ms
40 ms
50 ms
60 msRun A, t=2
Run B, t=15
Run B, t=20
11
Simulation time
Executable A
for (;;) {Run_A();Simulation.Sleep(1 ms);
}
Executable B
for (;;) {Run_B();Simulation.Sleep(5 ms);
}
SimulationTime Controller
Sleep 1 Sleep 51 ms passed 5 m
s passed
12
Tested in Development Environment?
Can:• High level functionality
• Multi-tasking
• Distributed systems
• Communication of parts
• Communication stress
Cannot:• Low level functionality
• Real concurrency
• Compiler differences
• CPU stress
13
SD
CABLESPAN 23tellabs
SD
OK1
OK2
PS1
PS2
TEMP
100
/15,2
30v~
20/
30Hz,
8.0/1
.75 A
BA
NK1
1x
2x3x
4x
5x
6x7x
8x
BAN
K21x
2x
3x
4x
5x6
x7
x8
x
1 5
2 6
3 7
4 8
BAN
K21 5
2 6
3 7
4 8
BAN
K1
CON
SOL
E
SD
OK1
OK2
PS1
PS2
TEMP
100
/15,2
30v~
20/
30Hz,
8.0/1
.75 A
BA
NK1
1x
2x3x
4x
5x
6x7x
8x
BAN
K21x
2x
3x
4x
5x6
x7
x8
x
1 5
2 6
3 7
4 8
BAN
K21 5
2 6
3 7
4 8
BAN
K1
CON
SOL
E
SD
OK1
OK2
PS1
PS2
TEMP
100
/15,23
0v~
20/3
0Hz,
8.0/1
.75 A
BAN
K1
1x2x
3x4
x
5x6x
7x8
x
BAN
K21
x2
x3
x4x
5x
6x
7x
8x
1 5
2 6
3 7
4 8
BAN
K21 5
2 6
3 7
4 8
BAN
K1
CON
SOL
E
SD
OK1
OK2
PS1
PS2
TEMP
100
/15,23
0v~
20/3
0Hz,
8.0/1
.75 A
BAN
K1
1x2x
3x4
x
5x6x
7x8
x
BAN
K2
1x
2x
3x
4x
5x
6x
7x
8x
1 5
2 6
3 7
4 8
BAN
K21 5
2 6
3 7
4 8
BAN
K1
CON
SOL
E
SD
TEST ACT ALM PWR1 2 3 4 C T R I LI OKNGLINE ALM
1 2 3 4LINE MODE
ACPWR
ON
OFFDC
1 21 2 3 12 3 4 5CLKMODE RATE
2.0A 2.0A
AC/DCTEST
OFF ON
SD
P ower
Fault
1 2 3 4 5 6
7 8 9 10 11 12
Link
Mode
Mode
Link
13 14 15 16 17 18
19 2120 22 23 24
Mode S elect
FdxAct
1X 2X 3X 4X 5X 6X
7X 8X 9X 10X 11X 12X
13X 14 X 15X 16X 17X 18X
19X 20 X 21X 22X 23X 24X
10/10 0Base-T Por ts
100Reset Clear
Module Stat usSelf Test
Console
P roCurve Switch 2424MHP J41 22B
14
User ExperienceLoad / StressExplorative
QA’s Cool New Role
Regression
Developer 1
Developer 2
Version Control Server
SubVersion(SVN)
Backed Up
Commit/Update
Commit/
Update
Update
Continuous Integration Server
CruiseControl
Not Backed Up
Uses
Build Scripts
Rakefile (Ruby Rake)
Target Binaries
Test reports
Monitors for feedback
Compile for PC simulation
Compile target binaries
Execute acceptance tests
Execute unit tests
Run test coverage
Continuous Integration Daily (fast, smoke suite)
Nightly (full suite)
15
12 releases later…
January May September
At some point
Development
To: 1 week/0-5 defects
From: 4 months/150 defects
Final System Test
Testing
16
Survey results (+5 .. 0 .. -5)
50%
15%
21%
14%
+5 (Much better)
+4
+3
+2
Key Remaining Challenges
Acceptance tests in low level language
Adopt new design style for testability
Flaky tests
17
Nightly build trend
0
200
400
600
800
1000
1200
1400
1 2 3 4 5 6 7 8 9 10 11 1213 14 15 16 17 18 19 20 21 22 23 24 25 26 2728 29 30 31Sprint #
Tes
t #
Acceptance Tests Unit Tests
23% line coverage
80% line coverage
18
Photo creditsRobot: beni_bbFire: winlordQuestion marks: immrchrisIce arrow: inyaFire Brigade: 13dede All @ stock.xchng
top related