iprg2012: a study on detecting modified peptides in a ... · pdf filedec. 2011) • 1...
Post on 13-Mar-2018
213 Views
Preview:
TRANSCRIPT
465.19 2+
464.59 3+
iPRG2012: A Study on Detecting Modified Peptides in a Complex Mixture A B R F
Proteome Informatics Research Group
John Cottrell1 , Karl R. Clauser2, Robert J Chalkley3, Ruixiang Sun4, Eugene Kapp5, Matt Chambers6, W. Hayes McDonald6, Henry H. Lam7, Nuno Bandeira8, Eric Deutsch9 and Thomas Neubert10
1Matrix Science, London, UK; 2The Broad Institute of MIT and Harvard, Cambridge, MA; 3University of California, San Francisco, CA; 4Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China; 5Ludwig Institute for Cancer Research, Parkville, Victoria, Australia; 6Vanderbilt University, Nashville, TN; 7Hong Kong University of Science and Technology, Hong Kong, China; 8University of California, San Diego, CA; 9Institute for Systems Biology, Seattle, WA; 10New York University School of Medicine, New York, NY
Study Goals
1. Evaluate ability of participants to identify modified peptides in a complex mixture 2. Find out why result sets might differ between participants 3. Produce a benchmark dataset, along with an analysis resource
Study Materials
• 5600 TripleTOF dataset (AB-SCIEX) – WIFF, mzML, dta, MGF (de-isotoped);–
conversions by MS Data Converter 1.1.0 – MGF (not de-isotoped – conversion by
Mascot Distiller 2.4)
• 1 FASTA file (SwissProt S. cerevisiae, human, + 1 bovine protein + trypsin from Dec. 2011)
• 1 template (Excel) • 1 on-line survey (Survey Monkey)
Study Instructions
1. Analyze the dataset 2. Report the peptide spectrum matches in the
provided template 3. Report measures of reliability for PTM site
assignments (optional) 4. Complete an on-line survey 5. Attach a 1-2 page description of your
methodology
How Much Do the Identif ications Overlap?
• Reasonable number of participants from around the globe, mainly experienced users but a few first-timers
• Large spread in number of spectra identified • False negatives (NS) are generally much higher than false positives, so there is
generally room for improvement • Peak list was a significant factor on performance • Varied performance in detecting PTMs
• Most participants struggled with sulfation • Multiply phosphorylated harder to find than singly
• Most common errors in site assignment were: • Reporting sulfo(Y) as phospho(ST) • Mis-assignment of site/s in multiply phosphorylated peptides
• The iPRG2012 are in the process of preparing the data for publication. If you participated and would like to help out, contact the iPRG through anonymous.iPRG2012@gmail.com.
For more information on the iPRG and for copies of this poster and the talk please visit: http://www.abrf.org/iPRG after the meeting
Acknowledgements: The iPRG are grateful to all of the participants. We would also like to thank Jeremy Carver (UCSD) for serving as the “Anonymizer”.
Prel iminary Conclusions
Study Design • Use a common, rich dataset • Use a common sequence database • Allow participants to use the bioinformatic tools and methods of their choosing • Use a common reporting template • Report results at an estimated 1% FDR (at the spectrum level) • Ignore protein inference
A Proteome Informatics Challenge
Nature uses a wide variety of protein post-translational modifications to regulate protein structure and activity and tandem mass spectrometry has emerged as the most powerful analytical approach to detect these moieties. However, modified peptides present special challenges for characterization. First, they are generally present at sub-stoichiometric levels, meaning that without enrichment strategies samples are dominated by unmodified peptides, so finding the modified peptides may be a challenge. Secondly, the modifications may have unique fragmentation behaviors in collision-induced dissociation (CID), which may need to be considered by database search engines. Finally, if there are multiple residues within a given peptide that could bear a particular modification type, then it is necessary to identify fragment ions that frame either side of the modification site in order to be able to localize the exact site of modification within the peptide. The Proteome Informatics Research Group (iPRG) created a collaborative data analysis study to enable proteomics laboratories to evaluate their ability to find a variety of post-translationally modified peptides within a complex peptide mixture background. The dataset consists of nearly twenty thousand high resolution and high mass accuracy tandem mass spectra. Within the sample there are peptides with a range of different natural and chemical modifications. This study enabled participants to evaluate their data analysis capabilities and approaches relative to others in analyzing a common data set, with a particular emphasis on their ability to detect and characterize peptides with modifications of potential biological significance.
Total Spectra vs. Interesting Mods
There is a very wide range in the total number of spectra with identified peptides. Once one focuses only on the spectra containing modifications for which the ability to localize the modification to a particular residue, the range is much narrower. The 5 rightmost participants went so far as to report only spectra of modified peptides.
Room for Improvement in ID Certa inty Thresholds
I d e n t i f i c a t i o n s r e p o r t e d a s : Ye s t h a t m a t c h e d t h e c o n s e n s u s ; N o , b u t s t i l l m a t c h i n g t h e c o n s e n s u s ; Ye s , b u t a d i f f e r e n t a n s w e r t h a n t h e c o n s e n s u s ; Ye s , < 3 c o n s e n s u s ; N o , t h a t d i s a g r e e d w i t h c o n s e n s u s
Table Key:
71755v 58288v 33564 93128i 11211 58409 87133i 94158i 97053i 42424i 77777i 23068 40104i 87048i 92653 34284i 23117 74564 14151 52781 47603 14152 45511 11821 Acetyl (K)
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Dimethyl (K) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Dimethyl (R) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Methyl (K) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Methyl (R) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Nitro (Y) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Phospho (STY)
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1
Sulfo (Y) 1
1 1
1 1 1 1 1
Trimethyl (K) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
An = Andromeda/MaxQuant MG = MS-GFDB pF = pFind Sc = Scaffold AS = A-Score MM = MyriMatch Pk = PEAKS SM = Spectrum Mill By = Byonics MO = MODa PkDB = PEAKSDB Sq = Sequest IH = In-house software O = OMSSA
Ppi =Protein Pilot ST = SpectraST IP = IDPicker Ot = Other PPr = Protein Prospector TPP = TransProteomic Pipeline M = Mascot P/PP = Pep/Prot Prophet PR = PhosphoRS XL = Excel Mde = Mascot Delta Score PD = ProteomeDiscoverer PW = ProteoWizard XT = X!Tandem Mdi = Mascot Distiller
71755v 58288v 33564 93128i 11211 58409 87133i 94158i 97053i 42424i 77777i 23068 40104i 87048i 92653 34284i 23117 74564 14151 52781 47603 14152 45511 11821
Peaklist Mgf mzML mzML mgf mgf mgf mzML mgf_nd mgf mgf_nd mgf mzML mzML mgf_nd mgf mgf mzML mgf mgf_nd mgf mgf mgf_nd mgf mgf_nd
mzML mzML mgf_nd WIFF
Spectral Pre-Processing Pk Pk PPi PPi MDi Ot SM pF Pk MDi PD PkDB PW PkDB Sq
Peptide Identification
By Pk Pk M PPi M O PPr pF M MG M P/PP SM pF M Pk M PPi M O M M PD PkDB PkDB O ST MM O TPP PkDB Sq
ST XT IH XT XT XT Ot
Discover Unexpected Mods By Pk Pk M PPi ST PPr pF IH MO SM pF M Pk PPi O M PkDB PkDB
Site Localization Pk Pk MDe PPi M PPr pF M IH M SM pF M Pk M AS IH An MDe PD
PkDB O MM O PkDB Sc Ot Sq ST IH
Results Filtering By Pk Pk IH PPi P/PP P/PP PPr pF IP IH XL IH XL pF M Pk M XL Sc M PR PkDB XL Ot TPP TPP XL PkDB
NTT 2 1 1 2 1 ? 1 1 1 2 1 ? 1 2 2 2 2 ? 1 2 2 2 ? 2
Experience 5-10 years 5-10 years 5-10 years >10 years < 1 year 3-4 years 5-10 years >10 years 5-10 years 5-10 years 5-10 years 3-4 years 5-10 years >10 years 1-2 years >10 years 1-2 years >10 years >10 years 1-2 years >10 years >10 years 1-2 years >10 years
1.48%
4.94%
6.1%
4.37%
n/a 13.68%
8.56%
2%
1.85%
16.67%
2.52%
n/a n/a
n/a
7.14% 2.92%
8.11%
3.45%
7.95%
2.52%
8%
3.03%
5.72% n/a
0
50
100
150
200
250
300
350
400
450
500
550
71755v 58288v 33564 93128i 11211 58409 87133i 94158i 97053i 42424i 77777i 23068 40104i 87048i 92653 34284i 23117 74564 14151 52781 47603 14152 45511 11821
# S
pike
d P
eptid
e PS
Ms
(FLR
%)
# Spiked PSMs Mod Loc Certainty N # Spiked PSMs Mod Loc Certainty Y, Ignored
# Spiked PSMs Wrong Mod Loc; Mod Loc Certainty Y # Spiked PSMs Correct Mod Loc; Mod Loc Certainty Y
Synthetic Peptide ID by Participant
Red corresponds to the presence of at least 1 PSM for a spiked synthetic peptide modified with the correct localization reported and the correct modification name. The localization certainty may have been reported as either Y or N. PSMs containing modification of residues other than s,t,y,k,r were excluded.
0
2000
4000
6000
8000
10000
12000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Cum
mul
ativ
e #
Spec
tra
# Participants Agreeing
Phospho vs Sulfo
Observe modified fragment ions.
Observe ‘unmodified’ fragment ions. Spectrum looks essentially identical to unmodified peptide spectrum
DISLSDY(Phospho)K
DISLSDY(Sulfo)K
Peak Processing • Two types of peak lists were supplied o Deisotoped (AB SCIEX MS Data Converter)
• Cannot infer fragment charge state • Possibly lower chance of false fragment matches
o Non-deisotoped (Mascot Distiller) • Can infer fragment charge state • Possibly higher chance of false fragment matches
• For 238 consensus spectra the peak lists had different specified charge state • 193 results only possible with deisotoped peak list • 45 results only possible with non-deisotoped peak list
0
1000
2000
3000
4000
5000
6000
7000
71755v 58288v 33564 93128i 11211 58409 87133i 94158i 97053i 42424i 77777i 23068 40104i 87048i 92653 34284i 23117 74564 14151 52781 47603 14152 45511 11821
# S
pect
ra
# No Mods # Common Mods (^q,^c,m,n,q) # Nterm Mods # AA Mutation Mods # Interesting Mods
0
100
200
300
400
500
600
700
# S
pect
ra
# Interesting Mod Loc Certainty N # Interesting Mod Loc Certainty Y
0 500
1000 1500 2000 2500 3000 3500 4000 4500 5000 5500 6000 6500 7000 7500
# Sp
ectr
a
#ND No Id, Diff from Consensus #Y<3 P Id Yes #YD Yes Id, Diff from Consensus #NS No Id, Same as Consensus #YS Yes Id, Same as Consensus
24 submissions from 23 participants. 9 were iPRG members. Participation was international and covered a wide range of experience level. Who Participated:
16
8
ABRF Member Nonmember
15 4
2 2 1
Bioinformatician/Software Developer Mass Spectrometrist Lab Scientist Director/Manager
13 4
4 2 1
North America Europe Asia Australia/NZ Africa 17
2
1 1 3 Academic
Manufacturer/Vendor Biotech/Pharma/Industry Government Other
5
6 11
2
Core Only
Software development only (no research facility) Conduct both core functions and non-core lab research Non-core research lab
Mixed Spectra Exposed by Peak Processing
464.59 3+ Non-deisotoped peaklist
465.19 2+ Deisotoped peaklist
top related