npt: potential uses in quantitative research tracy finch institute of health and society, newcastle...
TRANSCRIPT
NPT: Potential uses in Quantitative Research
Tracy Finch
Institute of Health and Society, Newcastle University
Objectives
To describe the development of a structured research instrument (questionnaire) to measure constructs of the NPM/T in relation to a particular intervention (E-health)
To consider wider issues for use of NPT in Quantitative research
Understanding Normalisation in relation to E-health
‘Understanding the implementation and integration of e-health services’. Mair, May, Finch, Murray, O’Donnell, Wallace et al. NHS SDO Funded study. (April 2006 – Jan 09).
Focused on barriers and facilitators to use of e-health systems by health professionals within the NHS
One work-package (out of 4) involved developing a structured research instrument to assess ‘readiness’ for e-health
The NPM and NPT framed data collection and analysis throughout project (Began with NPM – finished with NPT)
WP3 ‘Technology Adoption Readiness Scale’ (TARS)
Objective: To develop a structured, predictive instrument to test the contextual readiness of a health care setting for uptake and routine use of a specific ehealth system by health professionals.
Contextual readiness: combination of individual and organisational factors
Perspective: Professionals
TARS Study Structure
Phase 1: Instrument development included online survey of experts
Phase 2: Generic instrument ‘factors’ rated in importance
Phase 3: Specific instrument ‘questions’ about specific e-health system
being used
TARS Phase 1: Item development
Aim was to develop an initial item set, using expert review
Generate potential items through translation of construct statements into plain language and single dimensions
Item development: Example….
Step 1 Factor/ Issue:Concerns about security and confidentiality and standards
Step 2 Question:Does X affect your confidence in knowledge available to you? (Relational Integration)
Step 3 Question for ehealth study:How important are the following factors in affecting the use of e-Health systems in the everyday work of health professionals: ….how much the e-Health system affects the users’ confidence in their ability to conduct their work safely and efficiently (scale: Not at all important – Extremely important)
Phase 1: Expert Survey
Survey: 27 Items for inclusion (IW, RI, SW, CI) 5pt scale of ‘importance’ (+ ‘don’t know) Free-text space for ‘any factors missed’ Conducted electronically
Sample: Authors of reviews relevant to field
Recruitment: Email invitation
Phase 1: Results
Response: 63 completed surveys (25% of 252 invitations
presumed received)
Data analysis: Descriptive (mean ratings, correlations)
Findings: Importance of items generally highly endorsed Most correlations low-moderate (little redundancy)
and higher within than across NPM constructs Free-text responses useful in identifying further
factors to include
Phase 2: Development of TARS Generic
Aimed to: Refine factor set using expert survey data Test ‘relative importance’ from perspective of
professionals
Revisions to item set (31 Items): Some items re-worded, dropped or combined New items added from free-text responses
(phase 1) New items for NPT: Coherence, Cognitive Participation,
Reflexive Monitoring
Phase 2: Testing TARS Generic
Survey of health professionals’ perspectives of importance of factors affecting uptake and use of e-health
Sample: Regional NHS Hospitals Trust (potentially 10,000+ respondents). Extensive use of e-Health.
Recruitment: Via site contact (Technical Director)
Response: Extremely low (51 responses)
Analysis: Not particularly useful, but suggestive of different patterns of response between experts and professionals
Phase 3: TARS Specific
Aimed to: Develop a version containing questions framed for
an individual’s assessment of a particular e-health system
Test through data collection at (2) different sites:
Site 1: Community nurses using PDAs (relatively new)
Site 2: Established use of several e-health systems as basis of work (algorithms, information resources, etc)
TARS Specific Instrument
Demographics (eg. Professional role, system/s used, length of time using)
Comfort with using computer-based technology
TARS Items: 30 Items rated from ‘agree strongly’ to ‘disagree strongly’ (7 pt) (Items on hand-out)
‘Normalisation’ questions:
Whether system was ‘not at all’, ‘partly’, or ‘completely’ in routine use
Perceived likelihood of it becoming routine (5 pt scale)
Phase 3: Sample
SITE 1 % (n) SITE 2 % (n)
Response: 46/243 (19%) Response: 231/1351 (17%)
Age: 72% aged 45+; 29% aged <35 Age: 40% aged 45+; 29% aged <35
Sex: All Female Sex: 86% Female
Working role:Community Enrolled Nurse Community Staff Nurse District Nursing Sister/Charge Nurse Practice Development Nurse Senior Nurse
0 (0)28 (13)61 (28)9 (4)2 (1)
Working role:Call handlersNurse advisorsTeam leadersHealth Information advisorsOther
47 (109)24 (56) 9 (21)3 (7) 16 (38)
Perceived level of routinisation of e-Health Not at all Partly Completely
0 (0) 68 (30) 32 (14)
Perceived level of routinisation of e-Health Not at all Partly Completely
1 (2) 17 (35) 83 (174)
Non-parametric (cross-tab) analysis Groups perceiving e-health as ‘completely’ rather than
‘partly’ routine differed in expected direction:
on 12/30 Items at Site 1
(CI=4; RI=4; IW=1; Co; CP; RM
on 9/30 Items at Site 2
(CI=3; RI=3; SW=1; Co; RM) At Site 2, comparison of call handlers with nursing &
related staff indicated differences on 4 items
Key results: Normalisation Perceptions
Summary of Results of TARS
Development of NPM/T based questionnaire for assessment of individual’s perceptions of factors relating to normalisation of e-health Operationalising of NPM/T constructs into plain
language questions
Support for NPM/T in terms of constructs – patterns of relations between items
Potential of items representing NPM/T constructs for discriminating between levels of perceived normalisation of e-health
TARS - Limitations
Low response rates – insufficient for scale development work (statistical properties)
Constraints of ‘real’ environments:
‘Readiness’ assessment is dependent on timing and site characteristics
Lack of access to participants/control over survey reminders etc
TARS should be used/tested in further studies, in sites where predictive utility of TARS can be assessed prospectively
Using NPM/T in Quantitative Research: Wider issues
Potential Benefits
Challenges:
1. Translating theory into plain language
2. Addressing multiple perspectives
3. Standardisation vs specification
4. Operationalising ‘normalisation’
Summary
Using NPM/T Quantitatively:Potential Benefits
The ‘How much?’ question:
Structured surveys have the potential to collect data efficiently, and on a large scale
The ‘what is likely to happen?’ question:
Surveys, used prospectively, may have some predictive utility with respect to outcomes
Potentially useful in comparative research
Surveys are appealing to practitioners and researchers - facilitate take-up of the Theory!
Challenge 1: Translating constructs into plain language
Example:How important are the following factors in affecting the
normalisation of e-health…..“…… the extent to which organizational effort is allocated to an
ehealth system in proportion to the work that the system is intended to do” (CI)
Problem: Multi-dimensional constructs difficult to capture in single questions/statements
Possible solutions? Clear definitions of terms (eg effort) or Establish understanding of terms of reference (eg agreement on
what the system is intended to do?) and use several questions to build understanding
Challenge 2: Addressing multiple perspectives
Questions not to be framed around ‘intention’ – instead reflect judgements about others/the organisation
Which stakeholder groups should be included? How do we combine/weight their ratings?
Need for customising questions (or question sets) for different stakeholder groups
Challenge 3: Standardisation vs specification
Quantitative validation of NPT would be facilitated by: Focused effort on scale development in
appropriate settings and with adequate resources, and
Comparative analysis of quantitative research using NPT survey across different settings
However..... Can we develop a useful ‘generic’ NPT based
structured survey instrument that is useable across settings?
(and if we do, are we denying the complexity that the NPT embraces?)
Challenge 4: How to operationalise ‘normalisation’?
Does the NPM/T yet define ‘normalisation’ adequately for quantitative measurement of it as an ‘outcome’?
Are ‘perceptions’ of how much an intervention/technology/ practice has become ‘part of everyday work/life’ sufficient to test the constructs of the model?
i.e. Are ‘objective’ measures needed also?
Summary & Final Thoughts
Quantitative use of NPT brings challenges, but potential benefits are huge
TARS represents a useful starting point in developing quantitative use of the NPM/T
Need for more focused effort on scale development and validation (MRC Methodology Programme grant planned)
Other quantitative studies are underway (eg. May, Rapley et al. ‘BSPAR Survey’; Newton et al. Midwifery, Melbourne).