3. STARE-HI - Guidelines for authors ofIT evaluation studies

a) Why STARE-HI (Jan Talmon)

b) STARE-HI: Guidelines for authors

Motivation

• Good reports will be referenced

• Good reports have influence on the standing of the journal (IF)

• IJMI welcomes papers that evaluate HI in a clinical setting

Current Situation

• Variability in reporting

• Nearly all papers fall short on a few accounts

• Studies may be valid, but papers often raise more questions then being answered by the study

Main problems

• Status of system unclear

• Functionality of system unclear

• No account for sample size (power)

• Poor motivation for study design and methods chosen

• Poor discussion, no critical attitude

• Not clear what lessons are learnt

3. STARE-HI - Guidelines for authors ofIT evaluation studies

a) Why STARE-HI

b) STARE-HI: Guidelines for authors

Aim of STARE-HI

• STARE-HI = Standards for Reporting of Evaluation Studies in Health Informatics

• Provide guidelines on how to write an IT evaluation paper (a paper reporting on an IT evaluation study).

• To support• Authors when writing a paper• Reviewers and editors when assessing a paper

Development of STARE-HI (1/3)

• Only adaption of CONOSRT or comparable guidelines for RCT?

• Not really a solution, because• There is more than RCT • Socio-technical assessment• Qualitative studies• Specific issues of health informatics evaluaiton studies

Development of STARE-HI (2/3)

• Input for STARE-HI draft:

• Other recommendations such as CONSORT (RCT papers), STARD (studies of diagnostic accuracy), INAHTA (HTA reports), QUORUM (meta-analysis) etc.

• Own experiences as authors, reviewers and editors

Development of STARE-HI (3/3)

• Writing team of IT evaluation experts • EFMI WG• IMIA WG• AMIA WG

Structure STARE-HI

• Describes items that should be contained in the various sections of an IT evaluation paper

• Title and Abstract• Introduction• Method• Results• Discussion • Conclusion

Content of STARE-HI

• 1. Title• 2. Abstract• 3. Keywords• 4. Conflict of Interest• 5. Introduction

– 5.1 Scientific background– 5.2 Rationale for the study– 5.3 Ojectives of the study

• 6. Study context– 6.1 System details– 6.2 Location– 6.2 Study constraints, conditions and

context • 7. Method and material

– 7.1 Study design/method description– 7.2 Frame of reference– 7.3 Participants

• 7. Method and material (cont)– 7.4 Study duration– 7.5 Outcome– 7.6 Data acquisition– 7.7 Data analysis

• 8. Results– 8.1Baseline data– 8.2 Study flow– 8.3 Unexpected events– 8.4 Outcome data

• 9. Discussion– 9.1 Discussion of Findings– 9.2 Discussion of Methods

• 10. Conclusion• 11. References• 12. Appendices

How to proceed

• Discussion at MIE2006• Revision 1• Discussion through EFMI-WG/IMIA-WG website• Revision 2• Discussion at AMIA2006• Solicit comments of editors of MI and general medicine

journals• Revision 3• Final round for comments• Final version• Submission to MI and general medicine journals