TPP:6.0.0 Release Notes
From SPCTools
Trans-Proteomic Pipeline (TPP) software, release 6.0.0 is a major release of TPP that contains new features, improvements, and fixes to 5.2.0. The software is available for Windows as well as Linux (and unofficially) for OSX from all the usual locations (please see the section below, "Getting the software"). Most users are recommended to use the Windows installer, which installs and configures the TPP and other required software (such as a web server). For advanced users who need to customize the TPP, or for those who run on Linux or OS X, you can download the source code. Again, most users don't need build from source, and should use the Windows installer.
For installation help, please see TPP:5.2_Installation.
Previous release notes are available at TPP:5.2.0_Release_Notes
Contents |
Major Changes
- Added new TPP "Landing" page for local installation
- Mostly links to local TPP interfaces, external resources, release notes, and related info
- becomes new default location of TPP link (instead of Petunia)
- Added support for fetching remote spectra via their Universal Spectrum Identifier (USI) in Lorikeet
- Enables interactive examination and interrogation of spectra stored in public repositories (only PeptideAtlas so far)
- Can be displayed standalone or butterfly-style for comparison against currently loaded spectrum
- New tool fixPepXMLScan that can update (to a new file - no overwriting) pepXML files that have scan indexes that do not match the mzML files.
- New tool MikesMagicalMzMLShrinker to shrink mzML files.
- Its most basic functions remove white space and truncate user-defined identifier tags to a minimal size.
- More advanced features adds MS-Numpress compression to the files. No matter what, the data are zlib compressed (if they were not zlib compressed previously).
- Only works with mzML files. Not for mzXML. Not for mzML.gz. However, for mzML.gz, you can gunzip it, magically shrink it, then gzip it back up.
- New utility msnExtract for extracting fragment spectra
- New tool PepExtract
- simple single command tool to run comet, PeptideProphet, iProphet, PTMProphet and then export results as xls
- tpp/bin and tpp/cgi-bin have to be in the PATH
- New tool SPACEPro to quantify and evaluate digestion efficiency at the PSM, peptide, and protein levels.
- New utility StPeter2Matrix for combining multiple StPeter analyzed ProtXML files into a single 2-dimension data matrix of protein abundances.
- The application reconciles differences in protein grouping that occur among the different ProtXML files, and error rates (at least for now) are controlled using a single ProteinProphet analysis of all the combined datasets.
- New Tools: mzTrace, rtExtract, DidIScanThat, DIALib-QC (v1.2), QuanticProteinParser
- MADCAPS replaces old comet-fastadb.cgi as the default protein sequence viewer in TPP
- A link to the old viewer is still provided in MADCAPS
- New tool PPSX for Peptide-Protein Sequence eXploration. Mostly a front-end to promast
- New tool ptm2plot -- extracts PTM peptides and site probs from pepXML and creates files for visualizing in ploTPP
- New tool quiddist to query peptide ID info and overlay it onto DidIScanThat output
- takes in extra (optional) list of pepXML files
- Launch this tool from Petunia instead of DidIScanThat
- Show results in TPPcel table, with links to PepXMLViewer for each scan with an ID
- New tool TPPcel to visualize sets of similar tables in TPP
- Can load data via a few flavors of json, TSV (first 2 columns only), name=value lines, and various params files
- Can display data from multiple source files side by side to allow comparison of values, including ability to see only rows with non-equal values
- Simple one-click bar chart mechanism
- Experiment labels used in table headers are user-editable
- Able to receive "postMessages" from other TPP tools which trigger auto-loading of a new file
- Ability to Copy-Table-To-Buffer-As-TSV
- Includes js FileBrowser for ease of data loading
- New Unimod helper utility to view and add Unimod mass mods via pop-up widget
- list can be multiply filtered by user-specified string(s)
- user chooses from potential sites
- value of input element is set to conform to parameters format (comet/msfragger, kojak, tandem) and field type
- highlights associated input text field
- link to Unimod entry
- Added custom unimod.json to enable above (generated via local script from unimod xml export)
- Used in Petunia (comet etc params) and Lorikeet (custom user peptide)
- New TPPFileChooser utility to browse/choose files or directories to view/load via pop-up widget
- Added to Pep3D, ploTPP, PPSX, PepXMLViewer, ProtXMLViewer
- Able to sort/filter by name, size, and date
- Keeps directory history
- Can be repositioned via mouse-drag
- Uses new cgi to serve directory listing via json
- New NeoMzMLParser API -- a new, object API for reading and writing mzML files. Also supports scan-by-scan iterative reading or writing.
- New NeoPepXMLParser API -- a new C++ API for reading and writing PepXML files. This is intended to *eventually* replace the libpepxml currently in TPP.
- New NeoProtXMLParser API -- a new C++ API for reading and writing ProtXML files. This is intended to *eventually* replace the libprotxml currently in the TPP.
- Updated Look and Feel for most interfaces and viewers
New Features
- Lorikeet
- Added ability to read and display spectral library entries alongside acquired spectrum (e.g. SpectraST)
- Initial "butterfly" dual spectra plotting code courtesy of Bai Mingze
- New button to remove butterfly spectrum entry
- Add interactive utility to add custom user interpretation of spectrum:
- Displayed on separate tab, able to animate
- Can enter sequence, charge state, and mass modifications
- Dynamically display m/z of new peptide as user enters sequence/charge/mods, and compare to acquired m/z, showing diff in Da and ppm
- New Unimod Helper to find mass mods
- No support for custom neutral losses in custom view
- Added ability to read and display spectral library entries alongside acquired spectrum (e.g. SpectraST)
- novor2xml
- Add option to write one FASTA pseudo-protein per peptide
- After conversion, call EnzymeDigestionParser to calculate NMC
- Special handling for terminal mods (format not as advertised in docs!)
- set protein name to "_UNMAPPED" as per TPP convention and add helpful description
- Preprend peptide sequence to UNMAPPED protein name in order to get better protein mapping stats within PepXMLViewer
- clarify errors and warnings; minor changes to output to ease file comparison
- Petunia
- Major updates for AWS / amztpp integration
- Added explicit control to start/stop amztpp (local background process)
- process must be started here in order to launch searches
- Added 6 options for EC type to launch, as well as max.instances and %utilization
- Show last 20 lines of status
- Enable deletion of files in S3 bucket
- Show recently completed AWS services
- Revamped look and feel
- Add links to (external) management consoles
- Show number of jobs queued in page title
- Cleaned up logic of queued vs finished jobs
- Specify output directory for AWS Comet searches
- Minor text changes for clarity, and warning of charges on AWS
- Added explicit control to start/stop amztpp (local background process)
- Added utility to Add a New File in File Browser via pop-up widget
- choice of: blank (text); comet, tandem, or various msfragger params
- user chooses name; file is first written in /tmp, then moved to target dir
- Added interfaces for Mike's Magical Mystery Tool (mzML shrinker), DIALib-QC, DidIScanThat, clips (indexing of PEFF files), SPACEPro (enzyme digestion efficiency and quant)
- Added link to display command-line usage of most major tools (via async call)
- Added "dry run" option checkbox to most tools: shows command line but does not execute it
- Add "re-run" commands link, with warning/confirmation alert
- Display Tool Name on mouse-over in main menu for most major tools (e.g. "StPeter")
- Add initial support for MSFragger
- Add option to use promast in xinteract and re-map database pages
- Add new de Bruijn option to decoy fasta generator, make it the default
- Major updates for AWS / amztpp integration
- PepXMLViewer
- Update to new L&F
- Add new js FileBrowser widget to assist in file loading
- Add tpp AlertBox for error messages
- Added PEFF-related fields, with options to filter and sort on:
- num_saavs_min : when mapping to several proteins, the peptide sequence with minimal number of aa subs
- num_saavs_max : when mapping to several proteins, the peptide sequence with maximal number of aa subs
- peff_mods : number of aa mass mods coming from PEFF
- Display full SAAV pattern for each mapped sequence when viewing expanded protein list
- Added new num_prots column, which can be zero if UNMAPPED; links to madcaps for alignment
- Added peptide length column
- fix ptm field name for PTMProphet MASSDIFFMODE analysis.
- extend to visualize PTMProphet MASSDIFF and ADDUCT localizations ... also allow filtering for specific adducts or massdiffs ...
- Add min/max Quantic filter and display additional columns for Quantic
- Add min/max scan filter
- Added Kojak e-value field
- Load inverse reduced ion mobility
- Updated "ions" link for SpectraST results - now displays query and library spectra in butterfly mode in Lorikeet
- PTMProphet
- new options EXCLUDEMASSDIFFMIN= EXCLUDEMASSDIFFMAX= to allow PTMProphet in MASSDIFF mode to localize mass differences on sufficient magnitude.
- new Option: STATIC mass difference search option
- decoyFastaGenerator
- option to generate de Bruijn decoys
- New Enzyme definition lysarginase
- Include comet 2021.01.0 and related conf file -- Please read the relevant Comet release notes, as some parameters have changed behavior
- Include Kojak 2.0.0 (alpha-1) and related conf file
- Include PSI-MOD.obo, which is required for comet+peff mod searches
- Include gnuplot 5.28 (Windows)
- Upgrade (strawberry) perl to 5.30.2.1 (Windows)
Improvements/Changes/Bug Fixes
- DISCO
- more peptides from boxcar and ion mobility (TIMS-TOF) support
- change MININTENS= option so the filtered intensity is in units of a multiplicative factor on the minimum intensity peak in the spectrum, values 1 and higher ...
- for each target only extract unique mz fragments (don't chase the same fragment.)
- adjust correlation thresholds based on number of points used to compute the correlation ...
- replace SCAN_WINDOW option with MAX_SCAN_WINDOW MIN_SCAN_WINDOW options to define all window sizes to test
- automatically modulate the scan window for calculation of correlations to boost IDs, tweak the correlations of reported peaks to improve pseudo-spectrum quality
- retry readFile up to 100 times if failing because some file systems error out when you ask them for a file only once ...
- get back good performance on Triple TOF by adjust the binarySearchDataForPeak function, also experimental pea similarity..
- correct an inefficient Spectrum allocation exposed when ion mobility bins are set high ...
- apply different background filter in each ion mobility bin for extended dynamic range
- some extra newlines for easier to read progress reporting
- add a version number and a reference in mzML so can be properly formed mzML
- TIMS_TOF related changes
- Greater reliance on mzParser (less use of pwiz)
- Better thread load balancing
- increased sensitivity by averaging
- Correct bug associated with MININTENS= parameter that filters out lower intensity peaks to speed up processing
- correct some bugs associated with low sensitivity for TIMSTOF data ...
- correct a bug with no spectra returned when MININTENS= was enabled on non-TIMS-TOF file...
- BUGFIX correct crash on empty data ...
- BUGFIX don't go past the ends of the run!
- BUGFIX return ability to process irregular DIA files (e.g. ones with different types of MS1 scans)
- BUGFIX wasting time at the end of the run by overrunning counter corrected
- iProphet
- Increasing multi-threading ability for iProphet
- NRS minimum value -15 as before ...
- BUGFIX prevent a single thread in iProphet multi-threaded mode from disabling and entire model when it cannot find data ... make sure there is no data in all threads before models are auto-disabled.
- BUGFIX NSM model was not working with THREADS>1
- Libra
- Set m_reporterFromMS3=false as default in case parameter is not present in condition file
- Thermo MS3 mode is called SPS, not SRS...
- Plug memory leaks
- Lorikeet
- Updated to latest (Dec.19.2019)
- Added masses for TMT-16 tags
- Change default fragment tolerance to 0.25Da
- enable highlighting neutral loss peaks associated with GA-Ac tag 172.075837 on Lysine
- Display % of total MS2 ion current matched/annotated
- Gray-out theoretical fragment ions in table that are outside of the m/z range of acquired peaks
- made it easier to click on checkboxes and radio buttons
- Added stand-alone Lorikeet MS/MS spectrum viewer html/js page (eventually use this instead of cgi-based page)
- Fixed bug whereby we were not always capturing peptide charge
- BUGFIX: Lorikeet was incorrectly calculating the cterm modification masses, this corrects it.
- No longer pass fake [0,0.01] and [2000,0.01] datapoints to set x-axis range; use minDisplayMz and maxDisplayMz instead
- mzParser
- Handle Ion Mobility Data
- Support for inverse reduced ion mobility
- Updated mzML reader to handle latest Numpress settings represented by newer cvParams.
- recognize all types of MSnumpress+zlib binary data arrays.
- Added new behavior where random-access reading of non-ScanNumbered mzML data (i.e. Sciex data) now returns the 1-based index position in the file.
- replaces old behavior that returned a number indicating it was the Nth spectrum read.
- Note, if the mzML file does not have an index, then the scan number is always the Nth read, even for random-access.
- when a spectrum contains frame= , don't trust the scan= value...frame= data is DIA data and the scan= information should not be used fro the scan number, using scan index instead.
- Remove crufty scanIndex-based spectrum lookup. Solely use faster implementation of scanNumber lookup.
- Added extra diagnostics to error message of RAMPface:readHeader().
- read precursor info from mzML
- change memory management for multi-threaded speed on windows
- Petunia
- Expanded Libra Conditions to 25 channels
- Added channel defaults for TMT-16
- Also added defaults for DiLeu-21 tags (no support for isotopic correction factors yet, as these are quite different from current paradigm)
- Only use modified masses when running Static Mod quant!
- Added TPPcel link in File Browser for .tsv files
- Add links to new TPPcel visualizer for json files from ProtXML and DidIScanThat
- File name is passed to TPPCel via postMessage so that data can be added instead of stomping existing view
- Added link to SpectraST web page a link to PWiz/msconvert page
- Pre-check "Accurate Mass Binning" for PeptideProphet
- Pre-check to disable NSP model in iProphet (to avoid users running it also in ProteinProphet)
- Default StPeter and PTMProphet options to hi-res (not low)
- Automatically scroll to bottom of command output sub-windows (jobs page)
- Added one-fval and neg-gaussian options to xinteract
- Respect choice of Output Directory when processing files individually via xinteract
- Harmonize quant options in iprophet page with those of xinteract
- Allow custom specification for speclibs location
- After downloading PAtlas speclib, link to directory, not specific speclib
- allow auto-refresh with URL parameter so don't have to keep clicking
- enable QuanticProteinParser option for ProteinProphet page
- Enable vertical scrolling of top menu (e.g. when window is small)
- Add button to disable mass model in PeptideProphet
- Allow negative values when validating integer values (e.g. num threads)
- GammaNeg also an option for Comet results
- On the xinteract page allow the user to specify known decoys separately from unknown decoys
- Pre-fill mod AA masses for XPRESS and ASAPRatio by guessing from peeking into pepXML file
- Adjust m/z window values for XPRESS and ASAPRatio to 0.05 (from 0.5)
- Added hints for MS2 mass tolerance for StPeter
- Open/close sections by clicking on header bar; removed checkboxes
- Remove references to mzXML, refer to mzML
- Added option to configure Petunia banner text (and tab title) locally
- Allow 7 PTMs to be specified for PTMProphet (instead of 5)
- Rearrange ASAPRatio options
- Removed unused MALDI (file conversion) option in xinteract page
- Removed Area ratio setting
- Removed all references to long-deprecated raw->mzXML converters (ReAdW, mzWiff, etc)
- Removed long-retired link to runsearch, and put a warning on the RunSequest page
- Remove deprecated and rarely used ProteinProphet options
- Remove never-used options for ICAT models
- Remove WINDOWS option from Disco (no longer required; can still be passed via advanced params)
- Re-ordered a few items in TPPTools menu
- Create new Databases sub-section; move relevant tools therein
- Move decoy validation tools under Utilities
- Pin file actions buttons at bottom of viewport to avoid having to scroll to bottom of page
- Change FileChooser background color to white with orange border
- Added warning text to iProphet/NSP model
- Improve validation of options for decoy fasta generator
- Minor text changes for clarity and grammar
- Force windows drive to be lowercase.
- Correctly pass multiple user directories to updatePaths (updated to comma-separated)
- Updated link to XTandem API
- replace downloadFilesArchive subroutine with one that works ... previous one was corrupting mzML files!
- Bugfix DNA2AA wrong reference page
- BUGFIX switching pipelines was causing crash
- Bugfix: to properly expand filtered Jobs table entries
- Fix not being able to read extra long lines in messages pane
- Expanded Libra Conditions to 25 channels
- PepXMLViewer
- Added link to chromatogram for label-free XPRESS analysis ("peak_area")
- Report num_prots as "2" for intra-protein XLs, and "1" for inter-protein ones
- Add separator to expanded protein lists for XL crosslinks
- Do not consider _UNMAPPED entries as part of "unique proteins" count
- Do not add (empty) protein links to _UNMAPPED entries
- Add mouse-over tips to SpectraST and PeptideAtlas links
- Disable Glyco B checkbox if NxS/T box is not checked
- added check for incorrectly specified modification position
- Clean up layout of PTMProphet filters
- Place quant scale in (fixed) header
- Replace "close menu" icon with more obvious red "X"
- Replace messages "!" with a more screen-friendly icon
- Use standard TPP footer
- Pop "Messages" tab only once when file cannot be reached for status check
- Remove filename's potential colon...ouch
- Enable vertical scrolling of top menu (e.g. when window is small)
- Add xlink type to link to models page
- Added test for no peptide sequence: do not add links etc
- Properly trim whitespace in input file name
- Restore spreadsheet link
- Clean up Quantic score code to fix export to ploTPP
- fix bug where certain saav=0 stats were mis-counted
- BUGFIX correct plotTPP file path bug on Windows (now working!) ...
- BUGFIX correct problem with PTMProphet Filter disappearing when new columns are added ...
- PTMProphet
- enable analysis on SpectraST pepXML files ...
- More multithreading for speed ...
- Do not include in/output filenames in options string (which only gets written as an a tag attribute in pepXML, which already contains such info
- write intermediate files to current directory instead of WEBSERVER_TMP, which was filling up too often
- use extended SpectraST::Peptide library that allows to define 'user tokens' that would override any definition in the static mass tables and are allowed to have different masses in different instances of the Peptide class (e.g. in different threads of PTMProphet analysis) ...
- support for Magnum adducts, allow evaluating adducts and massdiffs on modified positions, allow evaluating adducts and massdiffs at the same time as variable mods defined in MASSDIFFMODE, exclude mass range feature for MASSDIFFMODE, annotate modifications as massdiff, adduct or param...
- switch mutex to try_lock instead of lock/unlock cycle, which was causing crashes on gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
- Bug fixes: MASSDIFFMODE functional again
- BUGFIX MASSDIFFMODE wrong mod masses being reported...also use the same prior for MASSDIFFMODE as for EM=1 and EM=3 to help EM converge ...
- fix xml formatting issues due to multithreading, implement options OSCOREMODE and QUANTMODE
- fix some bugs related to flat PTM probabilities and EM=1 and EM=3 execution options.
- BUGFIX address bug exposed when multiple variable mods of different types, but occuring on same amino acids, are be present in one peptide.
- BUGFIX correct deadlock issue when msms_run_summary has zero spectrum_query tags ... NEW: threaded updateProbs routine
- correct come issues with multithreading and writing invalid pepXML files ... prepare for adduct masses as annotated my Magnum ...
- BUGFIX add any static mods on the n-terminal to the mass of the n-terminal
- correct some issues related to analyzing files without probabilities ... also correct a bug with ROC curve reporting.
- BUGFIX clear out old tags ...
- BUGFIX correct error with reading NIONS / CIONS user parameter ...
- fix bug dealing with c-terminal modifications.
- Bug Fix: correct combinations to not evaluate more peptides than necessary
- reSpect
- mzParser now works with scan number ... mzParser corrected by Mike exposed reSpect bug
- BUGFIX mainly correct issues with writing mzML files using pwiz ...
- StPeter
- Added new parameter to set minimum probability threshold regardless of FDR.
- Added peptide-level normalized spectral index score (per Henry Lam's request)
- clarifying output to the user.
- Fixed rounding bug when representing modification masses on peptides.
- tpp models
- Update to new L&F
- Use new common tpp.css (also enables sticky table headers!)
- Added support for multiple PeptideProphet models in same file (e.g. for XL analysis); also added score overlay of massdiff, isomassdiff, ntt, nmc, etc
- Add button to toggle 1% Zoom in MASSD models
- Handle new matchedions model
- Also plot sum of pos+neg models for each charge state
- Display all PTMProphet run options
- Fix bug in parsing certain PTMProphet models; do not capture data points with value = zero; enable auto-tickmarks for MASSDIFF models (which can now be open search)
- Fix PTMProphet sens/err models and table
- HardKlor
- Updated to support Ion Mobility
- use spectrum member variable to boost speed in multi-threaded setting where reallocs add up
- MSToolkit
- Updated to support Ion Mobility (TIMS_TOF)
- mzParser deal with Precursors in mzML to store isolation window info
- Improved memory management with fewer reallocations to improve performance in multi-threaded applications
- Make usage statements somewhat more readable (esp. xinteract)
Miscellaneous
- Standardize progress bar across various TPP components
- New ISB-inspired wait "spinner". Added so far to ASAPRatio, Pep3D, and PPSX
- comet params: Place full path to PSI-MOD.obo file, now in TPP distro
- PWIZ Updated to compile on newer versions of gcc in msys2 mingw64 with pthreads
- More descriptive error messages and feedback in various tools
- Fix various memory leaks, and clean up of old/commented out code
- Removed antique TPP model rendering utilities, old (pre-Lorikeet) plot-msms.cgi, and all references to them
- CPAN perl modules Statistics::R, Statistics::Regression, XML::Twig added/required
Known Issues
- None
Getting the TPP Software
- Download the TPP version 6.0.0 native windows installer (TPP_Setup_6.0.0.exe) from the Sashimi SourceForge project file release page:
http://sourceforge.net/projects/sashimi/files/
- Everyone is encouraged to read and contribute to our wiki, at
http://tools.proteomecenter.org/wiki/
- For guides to installing and using our software, please see our wiki:
http://tools.proteomecenter.org/wiki/index.php?title=Software:TPP
- For downloading the source code, please go to the following link:
http://sourceforge.net/projects/sashimi/files/ and find the 6.0.0 source code .tgz package
- Or, check out the code directly from svn:
svn checkout svn://svn.code.sf.net/p/sashimi/code/tags/release_6-0-0
- For building from source, please refer to the README and INSTALL files in src/ directory of TPP as well as this wiki.
Acknowledgements
The TPP Team: David, Luis, Mike, Eric, Jimmy, plus all other developers who contributed to this release from ISB. Thanks to developers and users from the TPP's user community who provided feedback and code contributions.
SVN Log
The SVN log of all changes since the previous release is available at TPP:6.0.0_SVN_Log.
