IHE International Meeting

Gazelle ProjectGazelle ProjectSteve Moore, MIR Steve Moore, MIR

Eric Poiseau, INRIAEric Poiseau, INRIA

IntroductionIntroduction

IHE is expanding

New domains New participating countriesMore companies involved

Need better testing process

Need for conformity testing Need for interoperability

4

Participation to the European Connect-a-thon Participation to the European Connect-a-thon

2001 2002 2003 2004 2005 2006 20070

50

100

150

200

250

300

11

3343 46

7563

0

18

57

74 79

99

120

0

35

100

135

180

250

280

0

DomainsIntegration profilActorsCompaniesSystemsParticipants

What we have learnedWhat we have learned

Our Experience :

8 years of connect-a-thon8 years of MESA toolsConnect-a-thon on 3 continents !

Need to distinguish between :

Conformance testing Interoperability testing

Objectives of the projectObjectives of the project

Provide IHE with a new set of test tools :

For more in depth testing during IHE connect-a-thon

For companies internal testingFor testing between companiesFor Healthcare Enterprise QA/Acceptance

tests

IHE International Meeting

Conformance TestingConformance Testing

Conformance Testing (1/2)Conformance Testing (1/2)

Is unit testing Tests a single ‘part’ of a device

Tests against well-specified requirements For conformance to the requirements specified in the

TF and the referenced standards Usually limited to one requirement per test.

Tests at a 'low' level At the protocol (message/behaviour) level.

Requires a test system (and executable test cases) Can be expensive, tests performed under ideal

conditions

Conformance Testing (2/2)Conformance Testing (2/2)

High control and observability

Means we can explicitly test error behaviour Can provoke and test non-normal (but legitimate)

scenarios Can be extended to include robustness tests

Can be automated and tests are repeatable

Conformance Testing is DEEP and NARROW

Thorough and accurate but limited in scope Gives a high-level of confidence that key components

of a device or system are working as they were specified and designed to do

Limitations of Conformance TestingLimitations of Conformance Testing

Does not prove end-to-end functionality (interoperability) between communicating systems

Conformance tested implementations may still not interoperate• This is often a specification problem rather than a testing

problem! Need minimum requirements or profilesDoes not test a complete system

Tests individual system components, not the whole• A system is often greater than the sum of its parts!• Does not test functionality

Does not test the user’s ‘perception’ of the systemStandardised conformance tests do not include proprietary ‘aspects’

Though this may well be done by a manufacturer with own conformance tests for proprietary requirements

IHE International Meeting

Interoperability TestingInteroperability Testing

Interoperability TestingInteroperability Testing

Is system testing

Tests a complete device or a collection of devicesShows that (two) devices interoperate

within a limited scenario !Tests at a ‘high’ level (as perceived by users)

Tests the ‘whole’, not the parts Tests functionality

Does not necessarily require a test system

Uses existing interfaces (standard/proprietary)Interoperability Testing is BROAD and SHALLOW

Less thorough but wide in scope Gives a high-level of confidence that devices (or components in a

system) will interoperate with other devices (components)

Limitations of Interoperability TestingLimitations of Interoperability Testing

Does not prove interoperability with other implementations with which no testing has been done

A may interoperate with B and B may interoperate with C. But it doesn’t necessarily follow that A will interoperate with C.

Combinatorial explosionDoes not prove that a device is conformant

Interoperable devices may still interoperate even though they are non-conformant

Cannot explicitly test error behaviour or unusual scenarios

Or other conditions that may need to be forced (lack of controllability)

Has limited coverage (does not fully exercise the device)Not usually automated and may not be repeatable

IHE International Meeting

Conformance or Conformance or Interoperability Testing ?Interoperability Testing ?

Conformance or InteroperabilityConformance or Interoperability

Both are Needed :

Complementary, not competitiveETSI : « While it is not absolutely necessary to

undertake both types of testing, the combined application of both techniques gives a greatly increased confidence in the tested product and its chances of interoperating with the other similar products

ETSI : European Telecommunications Standards Institute

IHE International Meeting

Gazelle Year 1 Gazelle Year 1 PlanPlan

Year 1 targetYear 1 target

Prototype evaluate the design measure our estimatesApplied to selected profiles but designed to work

for all (known ones)• Radiology Scheduled Workflow (SWF)• Laboratory Scheduled Workflow (LSWF)• Patient Identity Cross-referencing (PIX)• Audit Trail and Node Authentication (ATNA)

Year 1 development planYear 1 development plan1st Quarter

HL7 & Dicom conformance tools (selected profiles) Integration Statement Management Application Logging Subsystem

2nd Quarter

Web system for Interoperability and conformance Tests HL7 & Dicom data sets

3rd Quarter

Simulated Actors User Management

4th Quarter

Report and evaluation

Year 1 ResourcesYear 1 Resources

IHE Europe 2 FTE1 FTE Inria

IHE North America 2 FTE (RSNA-MIR) NIST (XDS) ? IHE Canada / Infoway ?

IHE Asia

IHE-Japan (Translation work, Spec review)

ConsequencesConsequences

Mesa Tools

NO NEW DEVELOPMENT LIMITED SUPPORT BY STEVE

Kudu

NO NEW DEVELOPMENT LIMITED SUPPORT BY ERIC

But Connectathon Management Continues !

IHE International Meeting

Questions ?Questions ?