what do screeners refer for checking? allan wilson lead biomedical scientist in cellular pathology...
TRANSCRIPT
What do screeners refer for checking?
Allan WilsonLead Biomedical Scientist in Cellular
PathologyAdvanced Practitioner in Cervical Cytology
BAC 24th October 2013
Plan for today
• What do screeners refer for checking?
• Monklands audit of checkers• National survey of checker
role• Interactive session
BAC 24th October 2013
Screeners referral decisions
• What do screeners refer to checkers?
• What influences screeners referral patterns?
• How do we monitor referrals?• Can we influence over-referral
and under-referral?
BAC 24th October 2013
Why do screeners refer slides for checking?
• Marked inflammation ?dyskaryosis• Bland nuclear enlargement• Atrophy• ?Koilocytes / HPV / dyskeratosis• ?GA / ?Endometrium• ?Small cells• ?degenerate only• Local issues
BAC 24th October 2013
Some ground rules.....
• Images are representative of slides• All slides changed from BNA to
negative by checkers are double screened
• All from Thinprep slides• All stained using imager stain• Limited biopsy confirmation • Our opinions on BNA Negative
could be wrong......
BAC 24th October 2013
Why do screeners refer slides for checking?• Marked inflammation
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
Why do screeners refer slides for checking?• Bland nuclear enlargement
– ?perimenopausal or post menopausal changes.
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
Why do screeners refer slides for checking?• Atrophy
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
Why do screeners refer slides for checking?• ?HPV / ?Koilocytes
BAC 24th October 2013
BAC 24th October 2013
3a
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
Why do screeners refer slides for checking?• ?glandular abnormality / ?
endometrium
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
Why do screeners refer slides for checking?
•Immature squamous metaplasia
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
Why do screeners refer slides for checking?• ?degenerate only
BAC 24th October 2013
BAC 24th October 2013
2aBAC 24th October 2013
Why do screeners refer slides for checking?
• Check small cells ?high grade
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
Why do screeners refer slides for checking?
• Other oddities........
BAC 24th October 2013
BAC 24th October 2013
What influences screeners referral patterns?
• Confidence levels• Personal issues• Fatigue• Previous performance• Ability• Training• Post –invasive audit or review
BAC 24th October 2013
Over referral v Under referral (false +ve v false –ve!)
• We tend to notice over referral more quickly as it has a “nuisance” value!
• Under referral is potentially more dangerous and more difficult to detect
• Usual balance between sensitivity and specificity
BAC 24th October 2013
How do we monitor referrals?
• Gut feeling?• Worksheets• Spreadsheets• LIMS• Do we have any idea of what a
“normal” referral rate is?• What should we do if a referral
rate is deemed to be too low?
BAC 24th October 2013
How can we influence screener referrals?
• Feedback!• More feedback!• Training• Setting targets, self audit• Selected slide reviews• Don’t just live with over or under
referral
BAC 24th October 2013
Monklands Checker audit
BAC 24th October 2013
What is a checker?
• Poorly defined• How do we appoint checkers?• How do we train?• How do we assess competence?• Variable role• Often overstretched (HPV roles)
BAC 24th October 2013
Monklands checker survey
• Four checkers (13-25 years experience)
• Asked to record categories of slides and numbers called negative or referred over an approximate 3 month period
• Reflective of rota system in operation
BAC 24th October 2013
Summary of results
• 348 slides in audit• 56 referred to medic/AP as BNA+
•36 BNA•16 Low grade dyskaryosis•3 High grade (moderate)•1 severe dyskaryosis ?invasive
• 52 reported as BNA+ by medic/AP• 4 reported as negative
BAC 24th October 2013
Monklands checker survey (2013)
Checker Slides checked
Signed out Referred to AP/medic
1 174 140 (80%) 34 (20%)
2 82 70 (87%) 12 (13%)
3 60 54 (90%) 6 (10%)
4 32 28 (87%) 4 (13%)
Total 348 292 (84%) 56 (16%)
BAC 24th October 2013
Monklands checker survey (2006)
Checker Slides checked
Signed out Referred to AP/medic
1 72 49 (68%) 23 (32%)
2 132 59 (44%) 73 (56%)
3 86 60 (69%) 26 (31%)
4 87 67 (77%) 20 (23%)
5 105 89 (84%) 16 (16%)
Total 482 324 (67%) 158 (33%)
Monklands checker survey
• Can we suggest referral or signing out targets based on these figures?
• Are there any outliers?• We simply don’t know if checker 2 is
cautious (or lazy!) or checker 5 is confident (or dangerous!)
• We need to do more work over a longer period and involve more labs
BAC 24th October 2013
What type of slides prompted most referrals?
Slides checked
Signed out Referred to AP?Medic
Inflammation 160 134 (84%) 26 (16%)Nuc. enlargement 84 63 (74%) 21 (26%)Atrophy 4 4 (100%) 0?HPV 13 9 (70%) 4 (30%)?GA 21 19 (90%) 2 (10%)Check small cells 30 27 (90%) 3 (10%)?degenerate 36 34 (94%) 2 (6%)TOTAL 348 290 (83%) 58 (17%)
BAC 24th October 2013
Screener analysisScreener referred Reported as
BNA+1 28 8 (29%)2 56 20 (35%)3 24 2 (8%)4 76 12 (16%)5 6 06 82 8 (10%)7 6 4 (66%)8 16 09 28 0
10 26 0
What can we learn from this?
• Use to focus training/updates/review sessions
• Reduce inappropriate referrals and encourage appropriate referrals
• Invasive and HG audit suggests no issue with checkers reporting referred slides
• Possible link between low referral rates and low high grade sensitivity.......
BAC 24th October 2013
National survey on checker role
BAC 24th October 2013
Who replied?
• Web based survey on BAC website• Survey Monkey• 98 responses• 76 laboratories• 85% of UK cytology labs
BAC 24th October 2013
Health warning on analysis
• Duplicate entries from same lab• How do you select the “correct
response”• Multiple responses when only one
choice requested• Accuracy of responses• The “doomed” lab
BAC 24th October 2013
Cervical Cytology labs in the UK
• England 71• Northern Ireland 4 • Scotland 9• Wales 4
•Total 88
BAC 24th October 2013
Do you have staff who perform a Checker role?
• Yes: 75 (99%)• No: 1 (1%)
BAC 24th October 2013
Which statement best describes their role in your department
• They screen all abnormal slides referred by primary screeners
•30 (39%)• They only screen difficult slides, ?
BNA’s, “please check dots” cases•10 (13%)
• Perform both of the above roles•43 (57%)
BAC 24th October 2013
Do your checkers also primary screen?
•Yes: 69 (91%)•No: 7 (9%)
BAC 24th October 2013
Do you set a minimum number of slides that they must screen?
•Yes: 38 (50%)•No: 31 (41%)•No response: 7 (9%)
BAC 24th October 2013
If you do set a minimum figure, what is that figure?
• Wide range of responses from 20 to 3750!
• 34 labs did not respond• Top three responses were:
•3000 15 (20%)•750 8 (11%)•1000 6 (8%)
BAC 24th October 2013
Do your checkers participate in rapid review/preview?
•Yes: 73 (50%)•No: 1 (1%)•No response: 2 (3%)
•
BAC 24th October 2013
Do you monitor the performance of checkers?
•Yes: 56 (74%)•No: 17 (22%)•No response: 3 (4%)
BAC 24th October 2013
If you do monitor checkersperformance what do you measure?
• Workload•48 (63%)
• Number of referred abnormal slides from screeners that they agree/disagree with
•22 (29%)• Number of referred abnormals from
checkers to consultant that are agreed/disagreed with
•26 (34%)
BAC 24th October 2013
Does your laboratory require a secondopinion on slides where the checker has changed the primary screener report from abnormal to negative?
•Yes: 59 (74%)•No: 17 (22%)
BAC 24th October 2013
If you do is it for:
• All such slides•31 (41%)
• For High Grade changes only•26 (34%)
• No response•19 (25%)
BAC 24th October 2013
IQC of checkers
If you give a checker a ?small cell slide, how would you know if one checker consistently called them negative and one consistently called them severe dyskaryosis?
BAC 24th October 2013
Audit trails and CPA………
CPA insist that we record who embeds tissue, who cuts sections and what batch number of Pap stains we use but do we record every opinion given on every slide?
BAC 24th October 2013
Over view of survey (1)
• Most labs use checkers to screen all abnormals and “? BNA’s”
• Most checkers also primary screen and participate in IQC
• Less than half of responding labs monitor the rate of slides changed from BNA to negative
BAC 24th October 2013
Over view of survey (2)
• A third of labs do not get a second opinion on slides changed from BNA to negative
• 41% of labs set no minimum workload level
• Wide variation among those labs that do set a minimum workload level
BAC 24th October 2013
Conclusions
• Support and recognition of checker role
• Ensure opinions are recorded• Consider setting targets based on
local practice and monitoring• A minimum check should be a
double screen of all slides reported as BNA and above by screener but changed to negative by checker
BAC 24th October 2013
Conclusions (2)
• Review monitoring systems for screener referral and checker reporting
• Monitor screener referral rates (especially low referral rates)
• Monitor BNA “turnaround” rates by checkers
• National guidance on referral rates???• We cannot (and never will) agree on BNA!
BAC 24th October 2013
Interactive session
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
BAC 24th October 2013
And finally........
BAC 24th October 2013
BAC 24th October 2013
•Thank you for listening
•Any Questions?
BAC 24th October 2013