results interop test 2014
TRANSCRIPT
Regional Seminar on MRTDs and Traveller Identification Management Madrid, Spain, 25 to 27 June 2014
Arnaldo Cremisini, fedpol Switzerland Holger Funke, HJP Consulting
Results Interop-Test 2014
Interoperability Test • Crossover Test • Conformity Test
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 2
Objectives of Interoperability Test • Test of Documents (Samples) • Test of Inspection Systems • Test of Test Tools • Test of Specifications
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 3
Participants • 31 Document Providers
– 18 Samples’ Sets from Countries – 13 Samples’ Sets from Industries
• Total of 52 different document samples (One or two sets) • 10 Inspection System Providers
– 11 Inspection Systems stations • 3 Test Labs for Conformity Testing
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 4
The Interop Test Room
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 5
Crossover Test
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 6
Crossover Test
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 7
Crossover Test Samples • Mismatches between EF.CardAccess and DG14 (i.e. declared algorithms) • Some EF.CardAccess contained additional or unexpected information • Open questions on use of extended length (specification and support by IS
and samples) General • The quality of the used certificates varied widely (CSCA, DS and CVCA)
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 8
Crossover Test Inspection Systems • Some were upgraded during tests (end result after the tests: improved the interoperability) • Some were definitely not doing EAC and PA • Some were able to read the samples even if samples were not fully compliant (IS were
compensating for errors)
• Note that Integrated Mapping was NOT supported by all Inspection Systems • Not all algorithms were supported General • Make sure that IS support all algorithms
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 9
Crossover Test
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 10
Crossover Test
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 11
Crossover Test
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 12
Crossover Test
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 13
Crossover Test Notes • Almost all IS tested both BAC and PACE but some did not plan to support BAC
• In comparison with the Interop in London many more IS were supporting Integrated
Mapping • The expectations of the Interop session was that IS vendors would provide systems
representative of functional border control systems
• Not all samples were representative of Governmental issued eMRTD (some were more like development cards)
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 14
Crossover Test Some more statistical information BAC • 80% of the samples have been successfully read by all IS with BAC • but only 45% of the IS could read all samples with BAC SAC • 63% of the samples have been successfully read by all IS with SAC(PACE) • but only 55% of the IS could successfully read 98% of samples with SAC(PACE) • note that 1 IS could successfully read all samples with SAC(PACE)
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 15
Conformity Test • 3 Test Labs with Conformity Test Tools
– Keolabs (France): „ICAO Conformance Solution“ – TÜViT + HJP Consulting (Germany): „GlobalTester“ – UL (Netherlands): „Collis eMRTD Test Tool“
• Subset of „ICAO TR RF Protocol and Application Test Standard for e-Passports, Part 3“ Version 2.01:
– Test suite ISO7816_O: Security conditions for PACE protected MRTDs – Test suite ISO7816_P: PACEv2 – Test suite ISO7816_Q: SELECT and READ file EF.CardAccess – Test suite LDS_E: Matching between DG14 and EF.CardAccess – Test suite LDS_I: Structure of EF.CardAccess
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 16
Document Information (1/2) • Generic Mapping vs. Integrated Mapping* • SAC may use either IM or GM to map the nonce
– Samples supporting GM: 34 – Samples supporting IM: 7 – Samples supporting GM and IM: 5
• Additional in 2014: Chip Authentication Mapping *Based on 46 ICS
GM
IM
Both
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 17
Document Information (2/2) • PACE with
– ECDH: 39 – DH: 5 – Both: 2
• Number of PACEInfos – One PACEInfo: 36 – Two PACEInfos: 6 – Four PACEInfos: 4
ECDH
DH
Both
1 PACEInfo
2 PACEInfo
4 PACEInfo
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 18
Overall Test Results (Conformity) • Number of test cases performed: 21.282 • Results:
– Passed: 9.203 (13.925) – Failed: 615 (713) – Not performed: 4.514 (6.644)
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 19
Failed Test Cases per Document
0
10
20
30
40
50
60
70
80
90
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 20
Test Cases with failures (Top 7) Test Case #Fail Description LDS_I_01 20 Test of ASN.1 encoding of security infos in EF.CardAccess
LDS_I_03 20 Test of ASN.1 encoding of PACEDomainParameterInfo
7816_P_75 19 Positive test without domain parameter reference (DO 84) and eMRTD supports only one set of domain parameters
LDS_I_02 18 Test of ASN.1 encoding of PACEInfo
7816_O_41 15 Accessing the EF.DG3 file with Read Binary. The test verifies the enforcement of SM after the PACE protocol has been performed successfully.
7816_P_64 12 MSE: Set AT command without data object 80
7816_P_13 11 General Authenticate to get the encrypted nonce command with an additional object data
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 21
Number of Failures per Test Case
0
5
10
15
20
25
LDS_
I_01
ISO
7816
_P_7
5IS
O78
16_O
_41
ISO
7816
_P_1
3IS
O78
16_P
_30
ISO
7816
_P_1
1IS
O78
16_P
_15
ISO
7816
_P_5
6IS
O78
16_P
_61
ISO
7816
_P_5
5IS
O78
16_P
_20
ISO
7816
_P_2
3IS
O78
16_P
_17
ISO
7816
_P_1
0IS
O78
16_P
_18
ISO
7816
_O_3
7IS
O78
16_O
_38
ISO
7816
_O_5
2IS
O78
16_P
_76
ISO
7816
_Q_0
4IS
O78
16_P
_49
ISO
7816
_Q_0
3IS
O78
16_P
_27
ISO
7816
_P_4
0IS
O78
16_P
_26
ISO
7816
_P_2
5IS
O78
16_P
_39
ISO
7816
_P_3
6IS
O78
16_P
_38
ISO
7816
_P_0
5IS
O78
16_P
_70
ISO
7816
_P_0
2IS
O78
16_P
_32
ISO
7816
_P_3
1IS
O78
16_P
_72
ISO
7816
_O_5
3IS
O78
16_P
_53
ISO
7816
_P_4
8IS
O78
16_P
_46
ISO
7816
_P_6
6IS
O78
16_P
_43
ISO
7816
_O_4
2IS
O78
16_O
_51
ISO
7816
_P_6
5IS
O78
16_P
_07
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 22
Observations Conformity Testing • Document quality varies from
– Close to Release State vs. Experimental State • Test results differ between test labs
– Quality process to identify deltas • Different interpretations of
– Padding in EF.CardAccess and EF.DG14 – Encoding of TerminalAuthenticationInfo in EF.DG14 – Use of DO 84 in PACE – Use of ParameterID in PACE when proprietary or standardized domain parameters are used
• Certificates for EAC protocol were missing or not usable • Use of Test Specification Version 2.01 (two test labs) and 2.06 (one test lab)
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 23
With special thanks to Alan Bennett, DFAT Cor de Jonge, justid Jeen de Swart, justid
Mark Stafford, Infineon Nicolas Meuwly, fedpol Philipp Bättig, fedpol Stefan Brandl, OeSD
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 24
Contact Details Arnaldo Cremisini
[email protected] Holger Funke
Results Interop-Test 2014, Arnaldo Cremisini, Holger Funke 27/06/2014 25