what’s new in anatomic pathology quality assurance? raouf e. nakhleh, md mayo clinic florida
TRANSCRIPT
What’s new in Anatomic Pathology Quality Assurance? Raouf E. Nakhleh, MDMayo Clinic Florida
Disclosure InformationRaouf E Nakhleh, MD
• I have no financial relationships to disclose
Objectives
• Discuss evidence based guideline development and their impact on the future from a QA perspective
• Discuss issues related to immunohistochemistry validation
• Discuss ongoing assessment of practicing pathologists
Why these topics
• Not new
• Subject to regulation (USA)
• Formalization of issues not previously addressed
Evidence Based Guidelines
• How does one judge good medical practice?
• Much easier in retrospect, particularly when there is a negative outcome.
• Literature is vast with conflicting knowledge
• In the past, informal evolution of practice
• Emphasis on local standards
• Today national standards
Evidence Based Guidelines
• In particular situation: • Guidelines provide a road map on how to
proceed.• Guidelines help us judge appropriateness of
care• From a practitioner’s perspective, easier to
know if you are meeting your obligations• From a quality assurance perspective, easier
to create measures for comparison
Evidence Based Guidelines – Examples
• In pathology• Cancer protocols and checklists
• Standard set by physicians• Reporting standards• Accreditation by Commission on Cancer
• Clinical Laboratory Improvement Act • Standard set by legal decree• Procedural and quality standards
Evidence Based Guidelines
• Cancer reporting
• 1980 report• Breast, right, mastectomy:
• Infiltrating ductal carcinoma• 1 of 20 lymph nodes positive • Margins free
Evidence Based Guidelines• INVASIVE BREAST CANCER• • Macroscopic• 1. Specimen (partial breast, total breast including nipple and skin, etc)• 2. Procedure (excision without wire‐guided localization, total mastectomy, etc)• 3. Lymph Node Sampling (no lymph nodes present, sentinel lymph nodes, etc)• 4. Specimen Integrity (single intact specimen, multiple designated specimens, etc)• 5. Specimen Size (greatest dimension in cm)• 6. Specimen Laterality (right, left, not specified)• Microscopic• 7. Tumor Size (size of largest invasive cancer, may also be based on macroscopic)• 8. Tumor Focality (single focus, multiple foci, etc)• 9. Skin extent (not present, not involved, invades dermis, etc)*• 10. Nipple extent (not present, not involved, DICS, Paget’s disease)• 11. Skeletal muscle extent (not present, present, etc)• 12. Ductal Carcinoma In Situ (DCIS not present, present, etc)• 13. Lobular Carcinoma In Situ (LCIS not present, present, etc)• 14. Histologic Type of Invasive Carcinoma (ductal NOS, mucinous, papillary, etc)
• 15. Nottingham Score Overall Grade (1, 2, 3, not graded)• 16. Tubular Differentiation (1, 2, 3, not graded)• 17. Nuclear Pleomorphism (1, 2, 3, not graded)• 18. Mitotic Count (1, 2, 3, not graded)• 19. Margins (involved by invasive cancer, involved by DCIS, etc)• 20. Lymph Nodes ‐ total number examined*• 21. Lymph Nodes ‐ number with micrometastases*• 22. Lymph Nodes ‐ number with macrometastases*• 23. Lymph Nodes 0 size of largest metastatic deposit*• 24. Tumor pT Stage (pTX, pT0, etc)• 25. Regional Lymph Nodes pN Stage (pNX, pN0, pN1a, etc)• 26. Distant Metastasis M Stage (not applicable, pM1, etc)• Ancillary Studies• 27. Estrogen Receptor Disposition (performed, not performed, etc)• 28. Progesterone Receptor Disposition (performed, not performed, etc)• 29. HER2 Disposition (performed, not performed, etc)• * If not applicable to the specimen, code as if the element was present in the report
Evidence Based Guidelines – Laws
• Physician Quality Reporting System• Tax Relief and Health Care Act of 2006• Incentive pay (up to 2% of eligible Part B) for
physician who report quality measures (pay-for-reporting)
• Voluntary program• 2015 becomes mandatory
Physician Quality Reporting System
• Colon Cancer grade and stage (2008)
• Breast Cancer grade and stage (2008)
• Barrett’s esophagus (2012)
• Prostate grade and stage (2012)
• HER2 determination in breast cancer (2012)
Accountable Care Organization (ACO)
• Payment of ACO’s rewards quality and efficiency rather than volume
• Monitor practice patterns and use performance data to improve the quality of care• CMS establishes performance measures• If met, then eligible for shared savings• Over time - higher standards
• Use evidence-based guidelines
Overview of Potential Quality System
Evidence based medicine
Practice Guidelines
Addition of standardsInspection, QA review and PT of an institutions
National or local QA review
Accreditation
Improvement
Clinical studies
Evidence Based Guidelines
• What’s in our present and future?
College of American Pathologists Pathology and Laboratory Quality Center
• The Center develops evidence-based guidelines and consensus statements related to the practice of pathology and laboratory medicine. Through them, we continually improve the quality of diagnostic medicine and patient outcomes.
College of American Pathologists Pathology and Laboratory Quality Center
• HER2, ER and PR before official Center creation• CAP/ADASP Consensus Statements on Effective
Communication of Urgent Diagnoses and Significant Unexpected Diagnoses in Surgical Pathology and Cytopathology (published)
• Validating Whole Slide Imaging Systems for Diagnostic Purposes in Digital Pathology(complete, not published)
• CAP/ASCCP Lower Anogenital Squamous Terminology (LAST) Standardization Project for HPV-associated Lesions (complete, not published)
• CAP/IASLC/AMP Molecular Testing Guidelines for Selection of Lung Cancer Patients for EGFR and ALK Tyrosine Kinase Inhibitors(near completion)
Guidelines in the works• Immunohistochemistry (IHC) Assay Analytic Validation
Principles• CAP/ASH Algorithm for Initial Work-up of Acute Leukemia• ASCP/CAP/AMP/ASCO Molecular Markers for the Evaluation of
Colorectal Cancer• CAP-ADASP Interpretive Diagnostic Error Reduction in
Surgical Pathology and Cytopathology• Bone Marrow Synoptic Reporting for Hematologic Neoplasms• CAP-NSH Uniform Labeling Requirements for Blocks and
Slides in Surgical Pathology• ASCO/CAP Guideline Recommendations for HER2 Testing in
Breast Cancer – Update to 2007 Edition• ASCO/CAP Guideline Recommendations for HER2 Testing in
Gastric Cancer
Selection Criteria for Center Topics1. Patient risk, patient safety and quality; issues that
affect patient care and quality outcomes2. Performance characteristics of assay and ability to
reach consensus. Do problems exist including high false positive/false negative rates and lack of uniformity in practice with significant ability to reach consensus?
3. Amount of available evidence to predict a clinical response to an agent or change a patient outcome
4. Guidelines are lacking but regulatory bodies or other professional organizations are trying to preempt specialty development of guidelines or approval/disapproval or show unusual interest
5. Adoption momentum of a particular assay. Consider the number of pathologists affected.
18
Selection Criteria for Center Topics6. Feasibility of significantly changing practice
of pathology or medicine (e.g., using Ultra Sound Guided FNA to change cytology practice).
7. Ability to collaborate with the right partner(s) in development of a Center product to facilitate development and acceptance, and improve overall probability of success.
8. The public's perception of the issue negatively affects the image of pathologists and the pathologists' role in medicine.
9. Evaluation of existing methods is obsolete or does not bring value in patient care
19
Submit and Select Ideas
Determine Scope and Form Workgroup
Research and Review
Solicit Public Comment
Complete Recommendations
Review and Approve
Publish and Implement
Maintain
The Center Process for Developing Guidelines and Consensus Statements
IHC Antibodies and FDA Classification
• Antibodies are medical devices • Classification base on patient safety• Class I
• Adjunctive diagnostic information • Class II
• Prognostic or predictive data • Separately reported• More stringent quality assurance
• Class III• Require FDA premarket approval
HER2 Guideline
• Provide the appropriate HER2 assay validation procedure• Validation with FISH or with another IHC lab
• Prescribe appropriate fixation time and documentation: 6 – 48 Hours
• Modifies scoring criteria to enhance specificity
• Ongoing competency assessment for pathologists
• Enroll in HER2 proficiency testing service
HER2
• Guideline introduced in late 2006
• Mandated in 2008
• LAP checklist standards related to HER2 effective in late 2008
• CAP sponsored survey to determine how well labs are able to comply (late 2008)
• Leads to insight of effectiveness of guideline (2010)
LAP HER2 Standards
• Properly validate HER2 assays
• Ensuring appropriate fixation
• use the ASCO/CAP scoring criteria
• Enforce HER2 proficiency testing
2008 HER2 Survey
• Arch Pathol Lab Med 2010;134: 728-734
• HER2 assay validation
• Concordance with FISH• 81% of Labs achieved 95% concordance for(-)• 73% of Labs achieved 95% concordance for(+)
• Concordance with another IHC LAB• 72% of Labs achieved 95% concordance for(-)• 68% of Labs achieved 95% concordance for(+)
2008 HER2 Survey
• Fixation time and documentation
• 86% made changes to address fixation
• 60% process specimens on the weekend
• Documentation is addressed in reports in 70% of laboratories
2008 HER2 Survey
• ASCO/CAP scoring criteria
• 84% using criteria
• Ongoing pathologists competency assessment
• 91% have a program in place
2008 HER2 Survey – Summary
• Most labs are meeting HER2 guideline
• Gaps still exist, particularly validation
• Deficiencies hopefully corrected over time with inspection cycle
• Follow up survey conducted at the end of 2011, currently under analysis
Validation
• What about other antibodies?• Gap in understanding of Validation in AP• Few documents address IHC antibody
assay validation
• Definition• Confirmation through a defined process
that a method performs as intended
31
Principles of Test Validation
• Use manufacturer’s instructions• Use high-quality test materials with
known target values• Test materials should be of similar type
as patient samples• Complete documentation, including
procedure and results• Lab director review/approve results
IHC Validation
• For cell markers, (e.g. keratin, actin, Melan A etc.) how many cases should be performed in validation study?
• Should you validate with another lab or another technique?
• What level of concordance is acceptable?
• Are there situations where validation is not necessary?
• What about cytology material?
IHC Validation – Example • New antibody introduction – Prognostic • Example: MIB-1 (Ki-67)
• Proliferation marker (estimate of mitotic count)• Antibody needs to work at spectrum of findings (0 –
100%)• Include tissues with various levels of proliferation
• Carcinoid tumors of lung (0-10%)• Typical carcinoid (0-3%)• Atypical carcinoid (2-10%)• Small cell carcinoma (25-90%)
• Validation cases should cover the spectrum of findings
Immunohistochemistry Validation Procedures and Practice: A College of American Pathologists Survey of 727 Laboratories
Lindsay Hardy, MD (Boston, MA)Raouf Nakhleh, MD (Jacksonville, FL) Jeffrey Goldsmith, MD (Boston, MA)Patrick Fitzgibbons, MD (Fullerton, CA) Richard Eisen, MD (Greenwich, CT)Mary Beth Beasley, MD (New York, NY) Rhona Souers, MS (Northfield, IL)
Methods• September 2010: Questionnaire
• 1064 surveys distributed with CAP proficiency test mailing (HER-2)
• 754 returned (Oct. 2010)• 27 excluded• 727 included
• 2 major sections• Non-FDA approved, non-predictive• Non-FDA approved, predictive other than
HER-2
Results: Non-predictive
Validation Procedures Percent
Laboratory has a procedure for validation of new antibodies? 68
Procedure specifies the number of cases to be used? 54
Results: Non-predictive
• 68% had a written procedure
• 86% validated the most recently introduced non-predictive marker (despite lack of procedures for some)
Results: Non-predictive
Validation Procedures 10th Pctl Median 90th PctlMinimum # Cases to validate new antibody: Positive 2 8 20 Negative 0 5 15 Total Cases 4 13 30
Validation Procedures 10th Pctl Median 90th PctlMinimum # Cases to validate new antibody: Positive 2 8 20 Negative 0 5 15 Total Cases 4 13 30Acceptable concordance (%): Positive 80 95 99 Negative 50 95 99
Validation Procedures 10th Pctl Median 90th PctlMinimum # Cases to validate new antibody: Positive 2 8 20 Negative 0 5 15 Total Cases 4 13 30Acceptable concordance (%): Positive 80 95 99 Negative 50 95 99
# Cases used in most recent validation: Positive 2 7 20 Negative 1 5 15
Results: Non-predictive
Most recent validation Percent
Weakly or focally positive cases were included 40
Results confirmed by running parallel at another lab (same tissue) 18
Cases were tested on multiple days (between-run precision) 53
Results: Predictive, excluding HER2
Predictive Markers:o ER o PRo ER/PRo CD117/C-KITo KI-67
o P53o P63o PMS2o EGFRo Other
Validation Procedures Non-Predictive
Percent PercentLaboratory has a procedure for validation of new antibodies? 46 68
Validation Procedures Non-Predictive
Percent PercentLaboratory has a procedure for validation of new antibodies? 46 68
Procedure specifies the number of cases to be used? 65
Validation Procedures Non-Predictive
Percent PercentLaboratory has a procedure for validation of new antibodies? 46 68
Procedure specifies the number of cases to be used? 65 54
Results: Predictive, excluding HER2
• 46% had a written procedure
• 75% validated the most recently introduced predictive marker other than HER-2
Results: Predictive, excluding HER2Validation Procedures 10th Pctl25th Pctl Median 75th Pctl90th PctlMinimum # cases to validate new Ab:
Positive 3 5 11 20 25 Negative 1 5 10 15 25 Total Cases 5 10 25 35 50Acceptable concordance (%): Positive 90 90 95 95 99 Negative 50 90 95 95 99
# Cases used in most recent validation:
Positive 2 5 10 20 30 Negative 1 3 5 13 20
Revalidation
Validation Procedures Non-Predictive
Percent Percent
Procedure specifies repeat validation for:
New lot of antibody 64 66 Antigen retrieval 80 71 Detection system 81 74 Instrumentation 74 74 Fixative 78 65 Tissue processor 55 49
Validation Procedures Non-Predictive
Percent Percent
Procedure specifies repeat validation for:
New lot of antibody 64 66 Antigen retrieval 80 71 Detection system 81 74 Instrumentation 74 74 Fixative 78 65 Tissue processor 55 49Procedure includes specifications for use with cytology? 42
Validation Procedures Non-Predictive
Percent Percent
Procedure specifies repeat validation for:
New lot of antibody 64 66 Antigen retrieval 80 71 Detection system 81 74 Instrumentation 74 74 Fixative 78 65 Tissue processor 55 49Procedure includes specifications for use with cytology? 42 37
Summary• IHC validation: 727 labs
• Validation was performed at higher rates than the availability of written procedures
• Labs are generally following procedures• Most meet designated minimum number of cases • Trend to slightly fewer
• Guideline is needed to assist in Predictive and non-predictive IHC
• Uncertainty as to the # of cases needed• Uncertainty as to when to revalidate vs. verification• Uncertainty RE validation in cytology
Ongoing assessment of practicing pathologists
• Why now?
American Board of Pathologists
• Time limited certification (10 yrs)
• The four part MOC process requires diplomates to submit documentation in the following areas:
• Part I – Professional Standing
• Part II – Lifelong Learning and Self-Assessment
• Part III – Cognitive Expertise
• Part IV – Practice Performance Assessment
Joint Commission Standards
• Ms.08.01.01 Monitoring Performance; The organized medical staff defines the circumstances requiring monitoring and evaluation of a practitioner's performance (FPPE)
• Ms.08.01.03 Use of Monitoring Information: Ongoing professional practice evaluation information is factored into the decision to maintain existing privileges, to revise existing privileges, or to revoke an existing privilege prior to or at the time of renewal
Professional Practice Evaluation
• Use of objective performance data in the granting and/or maintenance of practice privileges
Ongoing Professional Practice Evaluation – OPPE • On an ongoing basis for privileged practitioners
Focused Professional Practice Evaluation – FPPE • When a practitioner is first privileged• When new privileges are first granted to an already privileged
practitioner• When insufficient activity or “performance issues” are identified
for a privileged practitioner
Concepts
• Professional competency assessment• Is part of the peer review process
(protected)• Is not publicly reported or stored in
credentialing files• Is a tool for evaluation, not an adverse
action• Produces some data points for
assessment • Thresholds are minimum bars, not goals
OPPE – Six Core Competencies
• Patient care
• Medical/Clinical knowledge
• Practice-based learning and improvement
• Interpersonal and communication skills
• Professionalism
• Systems-based practice
OPPE - Ongoing Professional Practice Evaluation
REQUIREMENTS
• Evaluation must be ongoing – More frequently than annually
• Process must be clearly defined • What data (“metrics”) will be monitored• Who will be responsible for review of data• How often data will be reviewed• How performance data will be used for decision
making• How data will be archived and made available
OPPE - Ongoing Professional Practice Evaluation
POSSIBLE OUTCOMES
• Privilege being evaluated may be continued, limited, or revoked
POSSIBLE ACTIONS
• Continue privilege – no action required
• Suspend or revoke privilege because it’s no longer performed or required
• Initiate FPPE – Due to insufficient activity or identified “performance issues” with privilege
OPPE - Ongoing Professional Practice Evaluation
• WHAT METRICS ARE APPROPRIATE FOR PATHOLOGISTS?
• Metrics for core vs. delineated privileges• Core privileges – Privileges broadly applicable to the
majority of practitioners in a discipline• e.g. anatomic pathologists, clinical pathologists,
etc.• Delineated privileges – Privileges specific to a
subset of practitioners within a discipline• e.g. surgical pathologists, cytopathologists,
hematopathologists, microbiologists, etc,
OPPE - Ongoing Professional Practice Evaluation
• EXAMPLES OF METRICS FOR CORE PRIVILEGES IN AP OR CP• Patient care – Turnaround time for key reports• Medical knowledge – General CME credits, successful MOC• Practice-based learning and improvement – Participation in
SAMs• Interpersonal and communication skills – 360° evaluation
provided by clinicians, lab staff, etc.• Professionalism – Attendance at medical staff meetings• System-based practice – Membership on hospital or other
organizational committees
OPPE - Ongoing Professional Practice Evaluation
• EXAMPLES OF METRICS FOR DELINIATED PRIVILEGES IN SPECIFIC AREAS
• Patient care• Surgical pathology – Turnaround time for biopsy reports
Number or % amended reports IOC vs. permanent dx
discrepancies• Cytopathology – FNA success rate• Clinical pathology subspecialties – timeliness of
administrative and clinical reports (transfusion reaction)• All subspecialties – Peer review of new method validations
in relevant lab section(s)• Medical knowledge
• Each subspecialty – CME credits relevant to subspecialty
FPPE - Focused Professional Practice Evaluation
• DIFFERENCES FROM OPPE• More restricted in scope – “focused”• Episodic rather than continuous, with finite endpoint
• SIMILARITIES TO OPPE• Basic process must be pre-defined and consistent• Similar monitoring methodologies (review of records,
direct observation, monitoring practice techniques, discussion with others involved in care)
FPPE - Focused Professional Practice Evaluation
• CIRCUMSTANCES WHEN NEEDED1. When a practitioner is first privileged2. When new privileges are first granted to an
already privileged practitioner3. When insufficient activity or “performance
issues” are identified for a privileged practitioner
Thank You!