Terry's blog
From SPCTools
Contents |
October 27, 2008
Eric thinks it will be useful to apply Mayu after ProteinProphet to compute the FDR (?)
October 27, 2008
Meeting with Eric:
Major topic is how to best incorporate iProphet into the TPP. Should iProphet be run on each experiment (possibly multiple searches) individually, or on many experiments together? Results with HUPO data suggest the latter. With the former, false positives accumulate. Here are some steps to take toward figuring out how to make use of iProphet:
- Establish this practice: always run xinteract (PeptideProphet) with -E (experiment_label) flag. This will make it easier to adapt createPipelineInput.pl to process iProphet output.
- iProphet is memory consumptive, which makes it difficult to use on lots of data. Possible solutions: break up spectra into batches and run iProphet on each separately, have iProphet generate its models based only on a random set of MAXSPECTRA (~ 1,000,000) spectra.
- We want to lower the pepXML probability threshold for inclusion in PeptideAtlas. Currently it takes the user-specified threshold (usually 0.9) and discards all identifications with probabilities less than this in the pepXML file, even if the ProteinProphet adjusted probabilities are above the threshold, because the adjusted probabilities aren't that different. But now with iProphet, the ProteinProphet adjusted probabilities could be significantly different. So we don't want to pre-filter peptides with PeptideProphet probs < 0.9. Instead, we hard-coded createPipelineInput.pl to pre-filter using prob < 0.5.
- SpectraST now uses only PeptideProphet probabilities. Henry is willing to use ProteinProphet probs if we give them to him in a text file. When the time is right, I should email Henry about this.
We also talked about stuff that should improve PeptideAtlas quality aside from iProphet:
- step01 of the PeptideAtlas build pipeline optionally takes a spectrum library and removes identifications that don't appear in the library. We can make use of this by going all the way through step08 (spectrum library creation using SpectraST), then running the build pipeline steps 01-08 again, applying the filter. Should improve our results.
October 21, 2008
Yesterday's meeting with Eric:
We had wished to be first to publish a large, high-quality human plasma proteome. Mann beat us to it just this month. Mann published 697 very high quality IDs. We have about 2400 but they need to be screened. Would be nice to publish 2000. We have a greater variety of data. HUPO plasma proteome project published about 3000 proteins. Leader = Gil Omenn, VP of HUPO. Will visit here. HUPO plasma project phase 1 is now over, a volume was published including a paper by Eric. Now we're in phase II.
AMASS method for screening spectra based on continuity of identified ions: two approaches:
- incorporate scoring function into PeptideAtlas so we can see how it performs
- incorporate into a program which processes mzXML files and adds a value in the <search_score> field to the pepXML. David S. has already expressed some interest in allowing an additional arbitrary discriminant score as input to PeptideProphet, which he would then try to model and would use if it were discriminating. The AMASS score could be such a score. If David implements this general functionality, then the score would automatically be used to influence peptide probabilities.
Want to figure out which search engines already consider ID continuity so we don't double model. SEQUEST does not. X!Tandem? Ask Henry.
A-Score method, already used at ISB for figuring out which sites are actually phosphorylated, could be incorporated similarly (?)
October 20, 2008
Read AMASS paper on filtering spectra based on (a) match percentage of high-abundance ions, and (b) consecutivity / continuity of matched fragment ions. Did not have time to comprehend the formulae. Paper is not highly cited.
Am building a PeptideAtlas from iProphet results. Creating wiki page describing the process.
October 14, 2008
Learning what a bad spectrum identification looks like
Talked for about an hour with Jimmy Eng, trying to find out the features of a misidentified spectrum. We looked at the five decoys that made it into the human urine peptide atlas. He was easily able to tag each of them as a poor identification. Concepts:
- Red flag: low intensity ID'd peaks amidst high density of low intensity non-ID'd peaks. Jimmy would devalue/ignore those peak IDs.
- Red flag: no strong ID'd peaks in the higher mass range
- Red flag: gaps in consecutivity, especially in higher mass range and esp. for Y-ions
- For precursor ion of charge N, should see strongest peaks for fragments of charge <= N-1
- Unidentified peaks at -18 (water) and -28 (?) may be OK
- Want to see lots of strong ID'd peaks rising above the level of the non-ID's peaks
- One decoy had R at N-term (missed cleavage) and no RK at C-term (miscleavage). Must have been a semi-tryptic search. Such a combination of unconventional cleavages is unusual but not extremely rare. I asked Jimmy if he'd devalue the ID and he said, "No, the semi-tryptic search was done because such cleavage patterns are expected." However, I didn't suggest tossing out the ID, just devaluing it. I think devaluing it might be appropriate.
- PeptideProphet does not consider X!Tandem's expect score, which is based on the distribution of scores for a particular spectrum vs. entire sequence DB. Expect = # of peptides we expect to see at this [raw] score by random chance. One decoy had an expect=12 -- very high. PeptideProphet only looks at hyperscore (top hit) and next (next best hit) and does not consider the distribution.
- Low mass portion of spectrum tends to be more messy and less interesting
- We want big, high mass Y-ions
I learned the following tangentially:
- PeptideProphet does make use of LC retention time.
- We do expect to see strong peaks just to the left of the precursor m/z.
- Mobile proton theory: Cleavage via CID occurs near any positive charge (proton). When there is a basic C-term (KR) and a basic N-term (N-term always basic), a spare proton is free to roam the length of the peptide because it is equally attracted to both basic ends. This makes cleavage approximately equally likely at all positions. If a peptide lacks that balance, then cleavage will tend to happen closer to any basic residue.
- Enhanced cleavage N-terminal to proline --> strong Y-ion beginning with that proline
- SEQUEST already considers consecutiveness of peak IDs in its preliminary scoring
- Jimmy has tried peak picking by sliding a window across a spectrum, calculating mean/SD of peak heights, to separate signal from noise.
October 13, 2008
States et. al. paper (Nature Biotechnology 2005) offers a method for estimating protein ID confidence levels which incorporates protein length. Eric implemented this in ProteinProphet and reports results in analysis.out. This method gives similar yield to simply tossing out all singletons, but is perhaps more theoretically sound (sometimes Eric thinks it's just more complicated).
Plan:
- Incorporate Swiss-Prot annotations into PeptideAtlas as a way of learning how to program PeptideAtlas
- Incorporate PeptideProphet information (e.g. probabilities) into PeptideAtlas. When changing schema, also insert spaces for States et. al. metrics:
- expected number of false positives per protein
- likelihood of actual # of hits occuring by chance ("probability" = 1 - this)
- Figure out how to get this info into PeptideAtlas
- Reload yeast PeptideAtlas so that this info is viewable, and draw conclusions from the info.
Talked to Lik Wee this morning -- new postdoc in Martin lab, studying proteomics. His first assignment is to learn the different search engines. I explained my project to him. Analyzing spectra for (a) large unidentified peaks, (b) big gaps in peak IDs, and (c) low average peak intensity sounds like a viable approach for appropriately deprecating some PeptideAtlas IDs. Can I use a machine learning algorithm for this? Ask Nikita.
October 8, 2008
I keep thinking the date must be a little later than it actually is. Like, I thought today should be the 9th. At the same time, it seems not so long ago that 2008 felt like a very new year, and it was hard to imagine it would ever feel otherwise.
My task for the next 11 months here at ISB is to reduce the false positive rate in PeptideAtlas. We want to catalog as many proteins as possible while minimizing inclusion of proteins that aren't really in the sample. Eric Deutsch and I have had several discussions about this, starting at my interview in May of this year. This is a place for me to record what I've learned from Eric, and to record ideas of my own and those I've received from others.
Ideas for reducing the false positive rate in Peptide Atlas
- Discard singletons (proteins represented by only a single spectrum
- Require a much higher probability cutoff for singletons (e.g. 0.99 instead of 0.90)
- Require a much higher probability cutoff for all protein identifications (e.g. 0.99 instead of 0.90)
- Set a fixed FDR, say 0.1%, and set probability cutoffs accordingly
- Local FDRs should match the global FDR
- Use Henry Lam's SpectraST quality filter
- Cutoff of 0.99 for all nobs (cryptic incomplete note)
- Make use of this observation: the decoy estimated FDR is much smaller than that obtained by averaging the probabilities of all (peptide?) identifications and subtracting from 1. Suggests that decoy estimated FDR is too small, or probabilities are too small.
- Recalculate proabilities for short peptides based on ((# short decoys) / (total hits to short peptides))
- Do the search engines make use of LC retention times? If not, try.
- Look at peak intensity. Searches and TPP do not look at this.
Spectrum features to watch for
- Big unidentified peaks are bad (possible exception: peak just to left of precursor m/z -- confirm with expert)
- Consecutive identified peaks are good. Breaks in this are bad.
- Important identifications should be on big peaks, not small ones. Searches and TPP do not take into account peak intensity.
