development of health informatics performance indicators

1Developing eHealth indicators / Hannele Hyppönen27.8.2015

Development Health Care Performance Indicators for Health Information

Systems Hannele Hyppönen

Developing eHealth indicators / Hannele Hyppönen 2

Background for the eHealth indicator work• According to the law on electronic processing of patient data

(9.2.2007/159), 20§– THL is responsible for planning, steering and monitoring of

electronic processing and information management of patient data, national information services as stipulated by 14§ and common social and health care information resources

Terveyden ja hyvinvoinnin laitos vastaa sosiaali- ja terveydenhuollon asiakastiedon sähköisen käsittelyn, siihen liittyvän tietohallinnon, 14 §:ssä tarkoitettujen valtakunnallisten tietojärjestelmäpalvelujen ja yhteisten hallinnonalakohtaisten tietovarantojen käytön ja toteuttamisen suunnittelusta, ohjauksesta ja seurannasta. (21.12.2010/1227)

27.8.2015


Development of health informatics performance indicators

1. Defining the context (human and environmental) for measurement -> complex sociotechnical system

– Identifying key stakeholders– Defining the relevant areas or systems in question

2. Defining the goals for measurement -> documented goals often very generic

3. Defining methods for indicator selection and categorization -> often science-based

4. Defining the data, collecting, analysing and getting feedback -> commonly agreed, comparable variables for international benchmarking are rare

27.8.2015

Policy (e.g. eHealth policy) implementation monitoring is often done top-down. How to involve different stakeholder groups for bottom-up definition of what to

monitor, where and how ?


1. Defining the context: systems and stakeholders

27.8.2015

EHRsEHRsChallenges: the context is a complex sociotechnical system with Many actors, tools, rules and environments -change in one elementImpacts other elements

eHealth Interventions, their Mechanisms of impact on Different aspects of health care performance inDifferent environments of use need to be well defined – especiallyfor international benchmarking


2. Defining the goals – what to monitor

27.8.2015

Generic goal of health IT:to improve human health, healthcare services, and wellness of individuals and populations

Specific goals: Analysis of eHealth policies and Stakeholder priorities

Challenges: documented goalsOften very generic, differentStakeholders emphasize different goals (e.g. data availability for secondary use vs. usability and utility for clinical care)

Developing eHealth indicators / Hannele Hyppönen 627.8.2015

3. Selecting and grouping of indicatorsChallenges:Outputs and outcomes emerge only when a system is available and functionsas expected, is used andhas an impact on the clinical and/or Management processes

This takes often a long time, and there are simultaneousOther intervening factors

IT impacts on health care structures

Impacts on HC clinical and management processes

Impacts on outputs and outcomes of care


4. Defining, collecting, analysis of data currently:

1-2) eHealth and eWelfare in Finland: survey of health care ICT availability, usage rate and trends (CIOs of health and social care organizations) (2003, -07,-10, -14, -)

3) eHealth Usability and experienced benefits: Physician survey (2010, - 14, -)4) Citizen’s use and experiences of eHealth services (2014, -)

5) Intensity of use of the National Information Services per data and user type (log data, pilot 2015, project commencing)

6) Register-based monitoring eHealth benefits: case national eHealth services for citizens (project commenced 2015-)

=>cross-scientific and-organizational teams designing the surveys and defining the national log and register data, involvement of physicians, but not direct involvement of citizens in survey design. =>Nordic collaboration to harmonize key measures

Challenge to get suitable, reliable log and register data => development of logs and registers (e.g. AvoHilmo)

27.8.2015


Contents of user-surveys• Physician survey:

– (background variables +) 1) Technical functioning, 2) Ease of use, 3) Support for collaboration within own organisation 4) Support for collaboration between organisations, 5) Support for collaboration with the patient 6) Availability and usability of test results, 7) Usability of key patient data 8) Experienced benefits, 9) biggest problems 10) Giving feedback and participation in EPR development.

• Citizen survey:– (background variables +) 1) respondent’s health 2) use of services and

portals 3) usability and utility of portals 4) adequacy of services received 5) importance of e-service goals 6) challenges of e-service use 7) importance of getting listed services electronically in future 8) giving feedback and participation in e-service development.

27.8.2015


Citizen’s use of e-services in 2014

27.8.2015

Developing eHealth indicators / Hannele Hyppönen 1027.8.2015

Table 1 Most and least important barriers of e-service use Most important barriers (>30% of respondents agreed) Agree %

Personal contact cannot be replaced by e-services 63

Terms of use are too long and complicated 42

Electronic services are not accessible for disabled people 40

I cannot get thorough services without face-to-face contact 38

Least important barriers (<20% of respondents agreed) Agree %

I cannot get service in my mother tongue 6

I do not have a computer and internet access 13

Electronic services slow down access to services and care 15

Electronic services do not bring any added value 17

I do not have adequate technical skills for using the services electronically 18

Electronic services are difficult to use 19


Benchmarking eHealth usability in the Nordic countries

27.8.2015


Reporting and plan for the future• Currently main products are publications (print, pdf),

distributed to main stakeholders• =>Researcher-driven, hard to get feedback

• Future: – Nordic and OECD-collabortion to harmonize measures

continues, Common Nordic citizen survey variables -> 2017

– Participation in the evidence-based management-network led by the Ministry => development and coordination of generation and use of evidence in decision making

– Bringing the eHealth monitoring results as part of statistical reporting in Finland/ Nordic countries

– Dynamic, user-driven reporting of results online => user-centred design of the interface

27.8.2015


Thank you! Questions?Litterature:

– H. Hyppönen, E. Ammenwerth, N. de Keizer. Exploring a methodology for eHealth indicator development. Stud Health Technol Inform. 2012;180:338-42.

– P. Hämäläinen, J. Reponen, I. Winblad, J. Kärki, M. Laaksonen, H. Hyppönen, M. Kangas. eHealth and eWelfare of Finland. Checkpoint 2011. Report 5/2013, THL, Helsinki, Finland, 2013

– H. Hyppönen, A. Faxvaag, H. Gilstad, G.A. Hardardottir, L. Jerlvall, M. Kangas, S. Koch, C. Nøhr, T. Pehrsson, J. Reponen, A. Walldius, V. Vimarlund. Nordic eHealth Indicators: Organization of Research, First Results and Plan for the Future. Stud Health Technol Inform 192:273-7, 2013.

– H. Hyppönen, J. Hyry, K. Valta, S. Ahlgren. Electronic services in the social welfare and health care sector. Citizens' experiences and development needs. National Institute for Health and Welfare (THL), Report 33/2014. 109 pages. Helsinki 2014.

– H. Hyppönen, M. Kangas, J. Reponen, C. Nøhr, S. Villumsen, S. Koch, G.A. Hardardottir, H. Gilstad, L. Jerlvall, , T. Pehrsson, , A. Faxvaag, H. Andreassen, B. Brattheim, V. Vimarlund and J. Kaipio. Nordic eHealth Benchmarking. Status 2014. TemaNord 2015:539. Nordic Council of Ministers, Copenhagen, 2015

– eHealth and eWelfare of Finland. Check point 2014. Report xx/2015, THL, Helsinki, Finland. (in process)

27.8.2015

development of health informatics performance indicators

Health & Medicine