please stand by… the webinar clinical validation of … · clinical validation of cancer...
TRANSCRIPT
CLINICAL VALIDATION OF CANCER BIOMARKER SIGNATURES USING ARRAY TECHNOLOGY
to login to Twitter and send tweets
Facebook login
if you need help
shows speaker bios
download slides and more info
LinkedIn login
shows slide window
Change the size of any window by dragging the lower left corner. Use controls in top right corner to close or maximize each window.
What each widget does:
opens the Ask a Question box
Twitter login (#ScienceWebinar)
search Wikipedia
shows the audio media player
PLEASE STAND BY… the webinar
will begin shortly…
Webinar Series Science
Sponsored by:
Participating Experts:
Paul “Mickey” Williams, Ph.D. SAIC-Frederick Frederick, MD
Timothy J. Triche, M.D.-Ph.D. University of Southern California Los Angeles, CA
Brought to you by the Science/AAAS Custom Publishing Office
Webinar Series Science
12 September, 2012
CLINICAL VALIDATION OF CANCER BIOMARKER SIGNATURES USING ARRAY TECHNOLOGY
Clinical Validation of Cancer Biomarker
Signatures using Array Technology
P. Mickey Williams, Ph.D. Director of the Molecular Characterization Laboratory
Clinical Research Directorate Frederick National Laboratory for Cancer Research
Frederick National Laboratory for Cancer Research
Definitions • Analytical Validity:
– Assay analytical performance and accuracy
• Sensitivity
• Specificity
• Reproducibility
• Precision
• Clinical Validity: – Accuracy of the assay to identify a clinical condition or outcome
• Positive and Negative Predictive Value
• Clinical Utility – Does the assay provide useful clinical information
– Does the assay improve clinical outcome of a patient
Frederick National Laboratory for Cancer Research
Often Published Array Studies Do Not Find Their Way Into Clinical Practice
• Biological heterogeneity confounds verification – Cellular, tumor, patient
• Assay analytical variability – Within assay, between assays
• Specimen variability – Tumor content, necrosis, inflammation
• Platform biases: not all assays are the same • Effect size: not only about p-value but also the
magnitude of difference between groups
A lot of “noise” that blurs marker and outcome correlation and validation
Frederick National Laboratory for Cancer Research
IOM Report: Evolution of Translational Omics: Lessons
Learned and the Path Forward
• http://www.nap.edu/catalog.php?record_id=13297
• Best practices for Translating Omic Tests into Clinical Use
Frederick National Laboratory for Cancer Research
Questions to Consider • What is the assay’s intended use?
– Critical to clearly state clinical intended use before beginning validation studies
– Be certain specimens used in assay validation are similar to the intended clinical specimens (should be similar to signature discovery specimens)
– How will assay results be reported?
• As a number? As a recurrence score? Positive/Negative?
• How will report be used for clinical action?
– How will the assay be used in the clinic?
• Integral assay (results used for patient management)
• Integrated assay (patients get assay as part of study but no direct clinical action results)
Frederick National Laboratory for Cancer Research
Hurdles for Validation of Gene Signatures
• Specimens of appropriate type are often hard to obtain, BUT critical to success
• Specimens from a single clinical study may add bias to results, ideally a second study provides the best specimens for assay validation – Assess possible bias introduced when only a portion of
specimens in a study are available (site bias, size bias, etc.)
• Multi-gene signatures lead to over-fitting of discovery data – Employ statistical tools to minimize
• Technical and reagent lot to lot variability
• Performance standards can be difficult to define
Frederick National Laboratory for Cancer Research
Some Suggestions For Success Prior to Validation
• Clearly state the clinical intended use
• Identify acceptable specimens and assess biases
• Consider the assay as a system, beginning with removal of the specimen from the patient and ending with an assay report to a clinician, the system must be well defined in SOPs
• Assemble a multi-disciplinary team: pathologist, laboratorian (CLIA accredited), technical staff, biologist/bioinformatician, oncologist
• Develop a well mapped plan for validation
• Lock down SOPs
• Lock down instrument performance specifications and instrument validation procedures
• If possible, find a second site
Frederick National Laboratory for Cancer Research
Some Suggestions to Consider: 2
• Attempt to find biological explanation for the genes and signature and their correlation with clinical outcome
• If assay is intended for use in clinical study or for implementation as a test for a CLIA laboratory: – Seriously consider an informal discussion with FDA sooner rather
than later
– If assay results will be used to manage a patient or stratify in a study OR if specimens used will be obtained for the sole purpose of the assay, I recommend speak with FDA
• If intending to charge for the assay, consult a business/marketing person familiar with the Dx industry
• Determine if other tests address similar outcomes – If yes, you will want to think carefully why your test will be better
and design clinical study to demonstrate that superiority
Frederick National Laboratory for Cancer Research
Validation Testing Locked Assay System
• Analytical validation testing assay report: – Sensitivity
– Specificity
– Repeatability
– Testing assay performance standards
– Is assay fit for purpose i.e. intended use and specimens
• Clinical validation (best done with new untested specimens) – Does the assay report accurately correlate with intended clinical
outcome
• May consider comparing across laboratories for comparability of results/report, this permits transparency and strong support for assay validity
Frederick National Laboratory for Cancer Research
Predictive Value • Positive Predictive Value =
The proportion for people with a positive test result who actually have the condition or outcome
• Negative Predictive Value = The proportion of people with a negative test result who do not
have the condition or outcome
Frederick National Laboratory for Cancer Research
Case Study LLMPP’s Lymphoma Assay
• Research discovery reported in 2000 Nature Alizadeh et al. – Two distinct molecular sub-classes of Diffuse Large B-Cell
Lymphoma identified by non-commercial microarray
– These sub-classes named GCB and ABC had significantly different prognosis and/or response to CHOP therapy
Frederick National Laboratory for Cancer Research
Unsupervised Clustering of DLBCL & Purified “Normal” B Cells
GCB
ABC
Effect Size of biological difference was sufficient to permit unsupervised identification of 2 classes
Frederick National Laboratory for Cancer Research
GCB/ABC Provides Independent Prognostic Value to IPI
ABC/GCB IPI Lo IPI & ABC/GCB
Frederick National Laboratory for Cancer Research
What Happened Between 2000 and 2012? • Subsequent to many experiments examining the genes of
ABC and GCB sub-classes, there is now an understanding of the underlying biology of these sub-classes – ABCs appear to have constitutive B-cell receptor signaling
– Many ABC tumors also exhibit NFkB signaling, ABC’s appear to be similar to activated B-cells found in the periphery
– GCBs appear similar to germinal center B cells found in lymph nodes
• Knowing these biological differences and results from multiple synthetic lethal experiments, relevant and drug-able targets have been identified for the poor prognostic ABC sub-class
Frederick National Laboratory for Cancer Research
What Happened Between 2000 and 2012? • Multiple platform switches have occurred
– Affymetrix custom LymphoChip
– Affymetrix U133 plus 2
– IHC
• A expression algorithm established
• LLMPP beginning a 2 PHASE study – Phase 1 Feasibility: A locked protocol and algorithm based on
Fresh Frozen Data will be employed for testing of 100 blinded FFPET specimens (matching fresh frozen section already tested)
– Algorithm tweaks will be made and assay will validated (analytically)
– Phase 2: Test set, including specimens collected outside of LLMPP will be tested with locked and analytical validated assay
Frederick National Laboratory for Cancer Research
Parting Comments • Think about the intended clinical use early and often
• Lock the entire assay system and capture protocols with SOPs
• Build a strong inter-disciplinary team
• Know the weak points of the assay system and build in quality checks in order to prevent an assay from reporting a false results
• If intended for clinical use engage a CLIA accredited laboratory and speak early with FDA
Frederick National Laboratory for Cancer Research
Clinical Assay Development Program • NCI has initiated the CADP to provide services needed to
support the clinical development of research-grade assays
• 8 CLIA accredited laboratories are part of Clinical Laboratory Network and available to assist with providing services
• Applications accepted 3 times/year
• Services include but are not limited to: – Clinical assay development/optimization
– Generation of locked documented protocols
– Assay performance and fit for purpose testing
– Statistical support
– Specimen collection
• www.cadp.nci.gov
Frederick National Laboratory for Cancer Research
Disclaimer • Views expressed are my own and not necessarily those of the
Frederick National Laboratory for Cancer Research-SAIC, National Cancer Institute, NIH, or DHHS
Frederick National Laboratory for Cancer Research
ACKNOWLEDGEMENTS
• Affiliation: Dr. P. Mickey Williams, Director Molecular Characterization and Clinical Assay Development Laboratory (MoCha), Clinical Research Directorate/MoCha, Frederick National Laboratory for Clinical Research, Frederick, Maryland, 21702.
• Funding: This project has been funded in whole or in part with federal funds from the National Cancer Institute, National Institutes of Health, under Contract No. HHSN261200800001E. The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.
Sponsored by:
Participating Experts:
Paul “Mickey” Williams, Ph.D. SAIC-Frederick Frederick, MD
Timothy J. Triche, M.D.-Ph.D. University of Southern California Los Angeles, CA
Brought to you by the Science/AAAS Custom Publishing Office
Webinar Series Science
12 September, 2012
CLINICAL VALIDATION OF CANCER BIOMARKER SIGNATURES USING ARRAY TECHNOLOGY
Microarrays for the creation of diagnostic and prognostic
biomarker profiles
Timothy J. Triche, MD, PhD Children’s Hospital Los Angeles USC Keck School of Medicine
RNA Seq vs. Microarrays Parameter Microarrays RNA Seq
Starting material amount Nanograms, More, typically >100 ng
Starting material type Fresh, FFPE (>20 yrs) Fresh preferable, FFPE ?
Signal Semiquantitative Digital
Specificity Cross hybridization Ambiguous mapping
Genome Coverage Limited to known content Unlimited (potentially)
Isoform detection Limited Unambiguous
Sequence variants No Yes
Interpretation Routine Challenging
Cost <$1,000 COGs <$1,000, interpretation ?
Clinical utility FDA acceptable Unproven
Discovery potential Limited to content Unlimited
Research Potential Limited, utilitarian Greatest potential
Coding Genes Account for Just 1-2% of the Genome:
-protein encoding
?
Non-coding Genes Account for Most Transcription from the Genome:
miRNA sprcRNA
sncRNA
PASR ncRNA
Antisense
Introns
lncRNA
-protein encoding
Transcriptionally active
Splice Variants
Large-Scale Transcriptional Activity in Chromosomes 21 and 22 Philipp Kapranov, Simon E. Cawley, Jorg Drenkow, Stefan Bekiranov, Robert L. Strausberg, Stephen P. A. Fodor, and Thomas R. Gingeras. Science 3 May 2002
The majority of total nuclear-encoded non-ribosomal RNA in a human cell is 'dark matter' un-annotated RNA
Tissue RNA Distribution Tumor RNA Distribution
Kapranov, P, et al: BMC Biology, December, 2010
Non-coding RNA networks underlying cognitive disorders across the lifespan
Summary: ncRNAs are numerous, diverse, and essential for brain development and cognition
Irfan A.Qureshi and MarkF.Mehler, Trends in Molecular Medicine, 684
Human Exon Array: Derived from ENCODE RNA expression data
• 5 million features on array • 1.4 million RNA transcripts • 0.2 million mRNA exons • 0.2 million intronic/anti-sense transcripts • ~ 1 million non-coding RNA transcripts!
Lessons Learned #1:
Non-coding RNA is highly relevant to biologic function and produces better
biomarkers
Coding vs Non-coding RNA Expression in Ewing’s Sarcoma
•
•
AK057037
ENPP3
Non-coding RNA
Gene
= non-coding RNA
= annotated genes
Lessons Learned #2:
Microarrays can deliver data that is very comparable to newer methods like RNA
seq, at a lower cost and faster.
Relative RNA Transcript Abundance: 3 platform comparison
RNA Seq (long read) HuEx (microarray) RNA Seq (short read)
Lessons Learned #3:
Exon-level microarrays can detect clinically relevant splice variants
A - A 3 A 7 ERMS EFT OS Wilms NB
Tumor-Specific Gene Splice Variants: ASS1 in Translocated ARMS
Absent in ERMS on U133 data Splice Variant Seen on HuEx Array data
ASS1: Alternative Splicing Database
Lessons Learned #4:
Exon-level microarrays that span the genome can detect novel RNA
transcripts
Results
N.B.: Underlined transcripts chosen for 24 feature ncRNA meta-feature
32 exon ncRNA meta-feature composition
Candidate PSR Runs Identified (corresponding to prognosis)
RNASeq of 600 kb Transcript
But Aceview reveals a 230 kb transcript
RefSeq
Aceview
No evidence of TSS or discontinuous transcription over entire 600 kb region
Summary
• Gene expression arrays that include coding and non-coding RNA transcripts can generate diagnostic biomarker profiles that outperform those composed of coding genes alone.
• Most RNA transcripts found by RNA Seq are detected by whole-transcriptome microarrays at comparable relative expression levels.
• When cost and ease of analysis are considered, gene expression microarrays remain an attractive means of assessing global RNA expression in a clinical setting.
Sponsored by:
Participating Experts:
Paul “Mickey” Williams, Ph.D. SAIC-Frederick Frederick, MD
Timothy J. Triche, M.D.-Ph.D. University of Southern California Los Angeles, CA
Brought to you by the Science/AAAS Custom Publishing Office
Webinar Series Science
12 September, 2012
CLINICAL VALIDATION OF CANCER BIOMARKER SIGNATURES USING ARRAY TECHNOLOGY
To submit your questions, type them into the text box
and click .
Look out for more webinars in the series at: webinar.sciencemag.org
For related information on this webinar topic, go to: www.affymetrix.com/genomics
To provide feedback on this webinar, please e-mail your comments to [email protected]
Sponsored by:
Brought to you by the Science/AAAS Custom Publishing Office Brought to you by the Science/AAAS Custom Publishing Office
Webinar Series Science
12 September, 2012
CLINICAL VALIDATION OF CANCER BIOMARKER SIGNATURES USING ARRAY TECHNOLOGY