Msconvert Wine

From SPCTools

(Difference between revisions)
Jump to: navigation, search
Revision as of 00:39, 25 January 2011
JoeS (Talk | contribs)
(→Instructions for CentOS)
← Previous diff		 Current revision
Edeutsch (Talk | contribs)

Line 1: Line 1:
-Proteowizard's msconvert utility is a must have tool for converting MS/MS data in almost any proprietary MS/MS data run into almost any of the commonly used open formats. While this tool is available on both Windows and Linux, the ability to convert many common formats under Linux is unavailable due to the need to have vendor specific libraries (dlls). Fortunately it *is* possible to run the Windows version of msconvert under Linux using [http://www.winehq.org Wine], a open source compatibility layer.+'''Last updated: 2019-01-09'''
 +'''WARNING: As of 2019-01, the ProteoWizard developers report that the best way to run the latest msconvert with vendor format conversion on Linux is to use the official Docker image at:
-== Instructions for Ubuntu ==+https://hub.docker.com/r/chambm/pwiz-skyline-i-agree-to-the-vendor-licenses
-http://uec-images.ubuntu.com/releases/10.10/release/+The instructions below are kept for historical purposes, but if you want to install Wine that works with ProteoWizard yourself, it is suggested that you first refer to the official container's Dockerfile:
-Tested with 64 bit image+https://github.com/ProteoWizard/container/blob/master/Dockerfile
-ssh -X+
-: sudo apt-get update+'''
-: sudo apt-get install wine+
-: sudo apt-get install cabextract+
-: wget http://www.kegel.com/wine/winetricks+
-: chmod 755 winetricks+
-: ./winetricks winxp+
-: ./winetricks vcrun2008+
-: ./winetricks dotnet20+
-Download and install Proteowizard+'''WARNING 2: This content below has not been updated in a long time and is probably obsolete. Below is a well-documented page on something that did work once at a point in time, but ProteoWizard and its dependencies, the vendor libraries, Wine, and the underlying operating systems are all moving targets. Therefore, the success story related below will likely not work on more recent platforms without additional tricks, or may not work at all. This page should be viewed as a story about something that did work once, and could work again, but may be difficult to get working on your particular platform. Better to use the references above.'''
-Go to http://proteowizard.sourceforge.net/download.html using a web browser. Click on the artifacts link for the Windows x86 (or x86_64) build. When the TeamCity login page is displayed log in as guest using the link provided. Download the file with the tar.bz2 extension. Extract the package and move it to wherever you want to install it:+Proteowizard's msconvert utility is a must have tool for converting mass spectrometry data from almost any proprietary data format into open standards formats (primary mzML and mzXML). While this tool runs on both Windows and Linux, the ability to convert many common instrument formats under Linux is limited due to the need to have vendor specific libraries (dlls) installed. Fortunately it is possible to run the Windows version of msconvert under Linux using [http://www.winehq.org Wine], a open source compatibility layer for Windows -- ''though your mileage may vary''.
-mkdir pwiz # cause pwiz is rude and pollutes+The following are instructions on how to install and setup Wine under a variety of Linux platforms. These instructions where largely derived from several spctools-discuss emails and a large thanks goes out to the users who posted them (see references below).
-cd pwiz+
-tar xvf ../pwiz*.tar.bz2+
 +== Requirements ==
 +* You '''will''' need to use wine in 32bit emulation mode (basically a 32bit wine even if you are on a 64bit architecture). This is because the .Net and Visual Studio libraries are required and these libraries are installed using a 32 bit installer, even for the 64 bit versions of these libraries. Please also be aware that not all proprietary vendor libraries that are distributed with ProtoWizard are 64 bit hence the 64 bit version of msconvert cannot be used to convert most proprietary files.
-http://teamcity.labkey.org:8080/repository/download/bt83/24694:id/pwiz-bin-windows-x86-vc90-release-2_1_2464.tar.bz2+== Instructions for Ubuntu (Updated 1/9/2013) ==
 + 
 +The following instructions where developed on a Amazon EC2 instance using Wine 1.5.21 and ProteoWizard 2.1.4131 on the official Ubuntu 12.04 'precise' Amazon Machine Image (AMI).
 + 
 +{| style="border-collapse: separate; border-spacing: 0; border-width: 1px; border-style: solid; border-color: #000; padding: 4px"
 +|[[Image:alert.jpg|middle|left]]
 +|Please read the notes regarding potential issues following these instructions before executing.
 +|}
 + 
 +# Connect to the system you want to install on. If using ssh use the -X option to ensure display forwarding is set since parts of the setup will prompt you with GUI dialogs asking you to accept licenses.
 +#:ssh -X ubuntu@<hostname>
 +# Install the latest 32-bit version of wine
 +#:sudo add-apt-repository ppa:ubuntu-wine/ppa
 +#:sudo apt-get -y update
 +#:sudo apt-get -y install wine1.5-i386
 +#:sudo apt-get -y install winetricks
 +# Ensure wine has the correct architecture and set the location of the Windows emulation
 +#:export WINEARCH=win32
 +#:export WINEPREFIX=~/.wine {{pad|5em}}''Default. Choose your location''
 +# Use ''winetricks'' to configure the Windows emulation
 +#:winetricks -q winxp vcrun2005 win7 vcrun2008 vcrun2010
 +# Progressively install the .Net frameworks. While winetricks is suppose to install any older frameworks I found it works better if I did it manually starting from the oldest. Also see note below about sites blocking downloads.
 +#: winetricks -q dotnet20 dotnet20sp1 dotnet20sp2
 +#: winetricks -q dotnet30
 +# Continue installing the rest of the .Net frameworks
 +#: winetricks dotnet30sp1
 +#: winetricks dotnet35''May fail. Rerunning it a 2nd time worked.''
 +#: winetricks dotnet35sp1
 +#: winetricks dotnet40
 +# Download and install Proteowizard.
 +#: Go to http://proteowizard.sourceforge.net/ using a web browser. Click on the downloads link and select the Windows with vendor support option with the tar.bz2 extension. Extract the package and move it to wherever you want to install it.
 +#:: mkdir pwiz; tar xvf pwiz*.tar.bz2 -C pwiz
 +# Register dll (shouldn't be needed but...)
 +#: wine regsvr32 pwiz/MSFileReader.XRawfile2.dll
 +#: wine regsvr32 pwiz/CompassXtractMS.dll
 +# Run msconvert
 +#:wine pwiz/msconvert.exe
 + 
 +==== Issues/Problems ====
 + 
 +===== Winetricks fails to download a .Net installer =====
 + 
 +For certain .Net installers winetricks may fail when trying to download the packages. From what I can tell some download sites may be blocking access. Just follow the instructions provided by winetricks. If you happen to be installing on a Amazon EC2 instance you may need to download the files from a system outside the cloud and copy the files to the instance.
 + 
 +===== dotnet20sp1 install completed but reported missing file =====
 + 
 +You may observe the following message while installing .Net 20 SP1:
 + 
 + dotnet20sp1 install completed, but installed file ... /c:/windows/assembly/NativeImages_v2.0.50727_32/indexb.dat not found­
 + 
 +This appears to be a known issue that can occur with dotnet20sp1 (see [http://appdb.winehq.org/objectManager.php?bShowAll=true&bIsQueue=false&bIsRejected=false&sClass=version&sTitle=&sReturnTo=&iId=10166]). According to the report its harmless and can be fixed by creating creating an empty file as so:
 +<br>
 +<br>{{pad|5em}}mkdir -p $WINEPREFIX/dosdevices/c:/windows/assembly/NativeImages_v2.0.50727_32
 +<br>{{pad|5em}}touch $WINEPREFIX/dosdevices/c:/windows/assembly/NativeImages_v2.0.50727_32/indexb.dat
 + 
 +===== .Net 3.0 installation window "disappears" =====
 + 
 +If during the installation of .Net 3.0 the installation dialog disappears or appears to hang, look in your "system tray" for the installer icon. This will be the final finish dialog that will need to be clicked through to end the installation.
 + 
 +===== AB Sciex wiff files throw an exception =====
 + 
 +When trying to convert AB Sciex wiff files the following error is outputted by msconvert/wine:
 + 
 + [WiffFileImpl::getSampleNames] The type initializer for 'Clearcore2.Data.WiffReader.WiffSampleRun' threw an exception.
 + 
 +At this time msconvert under the wine emulator can't be used to convert AB Sciex wiff files.
== Instructions for CentOS == == Instructions for CentOS ==
-AMI: ami-01996f68+The following instructions where tested using ProteoWizard on a CentOS 6.3 Amazon Machine Image (ami-f0ab24c0). CentOS presents a bigger challenge as you'll need the latest version of Wine and at the time these instructions where written none of the CentOS repositories contained pre-built rpms for it.
-: rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm+# Log into the system
-: yum -y update+# Upgrade your system if it hasn't already been
-: yum -y install xauth cabextract fontconfig gcc44 rpm-build+#: sudo yum -y upgrade
-: rpm -Uvh http://rpm.pbone.net/index.php3/stat/4/idpl/14565045/dir/centos_other/com/flex-2.5.35-7.x86_64.rpm.html+# Install necessary packages for building and running wine
-: wget ftp://ftp.pbone.net/mirror/ftp.sourceforge.net/pub/sourceforge/h/project/hp/hphp/CentOS%205%2064bit/RPM/flex-2.5.35-7.x86_64.rpm+#: yum -y installgroup "Development Tools"
-: rpm -Uvh flex*+#: yum -y installgroup "X Window System"
-: wget http://downloads.sourceforge.net/project/wine/Source/wine-1.2.1.tar.bz2?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fwine%2F&ts=1295048793use_mirror=cdnetworks-us-1+#: yum -y install libgcc.i686 glibc.i686 glibc-devel.i686 zlib-devel.i686 libX11.i686 libX11-devel.i686 freetype.i686 freetype-devel.i686 freetype-demos.x86_64 freetype-devel.x86_64 libxml2.i686 libxml2-devel.i686 libxslt.i686 libxslt-devel.i686 libXcursor.i686 libXcursor-devel.i686 libpng.i686 libpng-devel.i686 libjpeg.i686 libjpeg-devel.i686 libtiff.i686 libtiff-devel.i686 openssl.i686 openssl-devel.i686 libXrender.i686 libXrender-devel.i686 cabextract
-: rpm -Uvh http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS/rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm+# Download the latest development version of wine and install
-: ./configure --enable-win64 CC=/usr/bin/gcc44+# Get winetricks
-log out log back in+#:wget http://winetricks.org/winetricks
 +#:chmod +x winetricks
 +# Install prerequisites
 +#:./winetricks winxp vcrun2005 vcrun2008 vcrun2010 dotnet40
 +# Complete steps 8 thru end above
 +== What Works? ==
-CC=/usr/bin/gcc44+The following table shows the results of testing Wine/msconvert with the various different instrument formats available to us. For each format the instrument output was converted using the same version of msconvert installed on Windows 7 and Wine. The program readmzXML was then used to dump a summary of the resulting mzML file and the values compared between systems.
-rpmbuild --define "_without_gstreamer 0" --nodeps+
-32bit on 64....+If you have any questions about the testing or have additional results to add please forward them to the spc-tools discussion mailing list.
 +<br>
 +<br>
-yum install freetype.i386+{| border="1" cellpadding="2"
-configure: OpenCL 32-bit development files not found, OpenCL won't be supported.+|+ Wine/msconvert Instrument Formats Tested
 +! Vendor !! Model !! Acquisition Software !! File Type !! Result !! Notes
 +|-
 +| ABSciex (Applied Biosystems)|| QQQ || Analyst || .wiff file
 +| <span style="color:red">Fails</span> || Throws exception "The type initializer for 'Clearcore2.Data.WiffReader.WiffSampleRun'"
 +|-
 +| Agilent || QQQ || MassHunter 2.1 || .d directory
 +| Success ||
 +|-
 +| Agilent || QTOF || MassHunter 2.1 || .d directory
 +| Success ||
 +|-
 +| Bruker || Apex, MicrOTOF|| ''unknown'' || .baf file
 +| Success ||
 +|-
 +| Thermo || LTQ Orbitrap XL || Xcalibur 2.4 SP1 || .raw directory
 +| Success ||
 +|-
 +| Waters || ''unknown'' || MassLynx 4.1 || .raw directory
 +| Success ||
 +|}
-configure: libhal/libdbus 32-bit development files not found, no dynamic device+== How can I install windows applications to be shared by multiple users under wine? ==
-support.+
-configure: libgnutls 32-bit development files not found, no schannel support.+
-configure: libv4l 32-bit development files not found.+
-configure: libgphoto2 32-bit development files not found, digital cameras won't+
-be supported.+
-configure: gstreamer-0.10 base plugins 32-bit development files not found, gstre+
-amer support disabled+
-configure: libcapi20 32-bit development files not found, ISDN won't be supported+
-.+
-configure: libmpg123 32-bit development files not found (or too old), mp3 codec+
-won't be supported.+
-configure: libopenal 32-bit development files not found (or too old), OpenAL won+
-'t be supported.+
-configure: WARNING: libxml2 32-bit development files not found, XML won't be sup+Wine does not currently allow sharing its Windows root (or "prefix") between users, due to the risk of
-ported.+registry corruption from running multiple wineservers simultaneously ([[http://bugs.winehq.org/show_bug.cgi?id=11112 bug #11112]]). This means you typically would need to go through and install each set of Windows applications for each user. If you have a lot of users this could become burdensome, however you can create a single prefix and then copy it for each user. (The default wine prefix is ~/.wine). This saves having to run the same installation instructions for each user.
-configure: WARNING: libxslt 32-bit development files not found, xslt won't be su+Another option is to write a wrapper script around the whole process. This wrapper script could copy a standardized wine prefix with all of the applications installed into a new temporary prefix owned by the user, set the environment variable WINEPREFIX to point to the temp prefix, run the wine command(s), stop the user's wine server, and then remove the temporary prefix.
-pported.+
-configure: WARNING: GetText 32-bit development files not found. Internationaliza+One last item, be careful about using NFS with wine. If your home directory is mounted using NFS and shared across systems you could experience some problems running wine on multiple systems in parallel. More information about wine's "windows" partition and NFS:
-tion won't be fully supported.+
-configure: Finished. Do 'make' to compile Wine.+:http://www.winehq.org/pipermail/wine-users/2010-July/076137.html
== References == == References ==
 +
 +: http://groups.google.com/group/spctools-discuss/browse_thread/thread/604745729ac0e541
 +: http://groups.google.com/group/spctools-discuss/browse_thread/thread/bb2a608ad31f77dd/868fab027488bd09?hl=en&lnk=gst&q=wine#868fab027488bd09
 +: http://groups.google.com/group/spctools-discuss/browse_thread/thread/2d98145b42479cb0/f8d80a25d1f8583c?hl=en&lnk=gst&q=wine#f8d80a25d1f8583c
 +: https://groups.google.com/d/msg/spctools-discuss/syqu3YBzzlg/tnSQIkCKhMQJ

Current revision

Last updated: 2019-01-09

WARNING: As of 2019-01, the ProteoWizard developers report that the best way to run the latest msconvert with vendor format conversion on Linux is to use the official Docker image at:

https://hub.docker.com/r/chambm/pwiz-skyline-i-agree-to-the-vendor-licenses

The instructions below are kept for historical purposes, but if you want to install Wine that works with ProteoWizard yourself, it is suggested that you first refer to the official container's Dockerfile:

https://github.com/ProteoWizard/container/blob/master/Dockerfile

WARNING 2: This content below has not been updated in a long time and is probably obsolete. Below is a well-documented page on something that did work once at a point in time, but ProteoWizard and its dependencies, the vendor libraries, Wine, and the underlying operating systems are all moving targets. Therefore, the success story related below will likely not work on more recent platforms without additional tricks, or may not work at all. This page should be viewed as a story about something that did work once, and could work again, but may be difficult to get working on your particular platform. Better to use the references above.

Proteowizard's msconvert utility is a must have tool for converting mass spectrometry data from almost any proprietary data format into open standards formats (primary mzML and mzXML). While this tool runs on both Windows and Linux, the ability to convert many common instrument formats under Linux is limited due to the need to have vendor specific libraries (dlls) installed. Fortunately it is possible to run the Windows version of msconvert under Linux using Wine, a open source compatibility layer for Windows -- though your mileage may vary.

The following are instructions on how to install and setup Wine under a variety of Linux platforms. These instructions where largely derived from several spctools-discuss emails and a large thanks goes out to the users who posted them (see references below).

Contents

[edit]

Requirements

  • You will need to use wine in 32bit emulation mode (basically a 32bit wine even if you are on a 64bit architecture). This is because the .Net and Visual Studio libraries are required and these libraries are installed using a 32 bit installer, even for the 64 bit versions of these libraries. Please also be aware that not all proprietary vendor libraries that are distributed with ProtoWizard are 64 bit hence the 64 bit version of msconvert cannot be used to convert most proprietary files.
[edit]

Instructions for Ubuntu (Updated 1/9/2013)

The following instructions where developed on a Amazon EC2 instance using Wine 1.5.21 and ProteoWizard 2.1.4131 on the official Ubuntu 12.04 'precise' Amazon Machine Image (AMI).

middle
Please read the notes regarding potential issues following these instructions before executing.
  1. Connect to the system you want to install on. If using ssh use the -X option to ensure display forwarding is set since parts of the setup will prompt you with GUI dialogs asking you to accept licenses.
    ssh -X ubuntu@<hostname>
  2. Install the latest 32-bit version of wine
    sudo add-apt-repository ppa:ubuntu-wine/ppa
    sudo apt-get -y update
    sudo apt-get -y install wine1.5-i386
    sudo apt-get -y install winetricks
  3. Ensure wine has the correct architecture and set the location of the Windows emulation
    export WINEARCH=win32
    export WINEPREFIX=~/.wine  Default. Choose your location
  4. Use winetricks to configure the Windows emulation
    winetricks -q winxp vcrun2005 win7 vcrun2008 vcrun2010
  5. Progressively install the .Net frameworks. While winetricks is suppose to install any older frameworks I found it works better if I did it manually starting from the oldest. Also see note below about sites blocking downloads.
    winetricks -q dotnet20 dotnet20sp1 dotnet20sp2
    winetricks -q dotnet30
  6. Continue installing the rest of the .Net frameworks
    winetricks dotnet30sp1
    winetricks dotnet35May fail. Rerunning it a 2nd time worked.
    winetricks dotnet35sp1
    winetricks dotnet40
  7. Download and install Proteowizard.
    Go to http://proteowizard.sourceforge.net/ using a web browser. Click on the downloads link and select the Windows with vendor support option with the tar.bz2 extension. Extract the package and move it to wherever you want to install it.
    mkdir pwiz; tar xvf pwiz*.tar.bz2 -C pwiz
  8. Register dll (shouldn't be needed but...)
    wine regsvr32 pwiz/MSFileReader.XRawfile2.dll
    wine regsvr32 pwiz/CompassXtractMS.dll
  9. Run msconvert
    wine pwiz/msconvert.exe
[edit]

Issues/Problems

[edit]
Winetricks fails to download a .Net installer

For certain .Net installers winetricks may fail when trying to download the packages. From what I can tell some download sites may be blocking access. Just follow the instructions provided by winetricks. If you happen to be installing on a Amazon EC2 instance you may need to download the files from a system outside the cloud and copy the files to the instance.

[edit]
dotnet20sp1 install completed but reported missing file

You may observe the following message while installing .Net 20 SP1: 

dotnet20sp1 install completed, but installed file ... /c:/windows/assembly/NativeImages_v2.0.50727_32/indexb.dat not found­

This appears to be a known issue that can occur with dotnet20sp1 (see [1]). According to the report its harmless and can be fixed by creating creating an empty file as so:

 mkdir -p $WINEPREFIX/dosdevices/c:/windows/assembly/NativeImages_v2.0.50727_32
 touch $WINEPREFIX/dosdevices/c:/windows/assembly/NativeImages_v2.0.50727_32/indexb.dat

[edit]
.Net 3.0 installation window "disappears"

If during the installation of .Net 3.0 the installation dialog disappears or appears to hang, look in your "system tray" for the installer icon. This will be the final finish dialog that will need to be clicked through to end the installation.

[edit]
AB Sciex wiff files throw an exception

When trying to convert AB Sciex wiff files the following error is outputted by msconvert/wine: 

[WiffFileImpl::getSampleNames] The type initializer for 'Clearcore2.Data.WiffReader.WiffSampleRun' threw an exception.

At this time msconvert under the wine emulator can't be used to convert AB Sciex wiff files.

[edit]

Instructions for CentOS

The following instructions where tested using ProteoWizard on a CentOS 6.3 Amazon Machine Image (ami-f0ab24c0). CentOS presents a bigger challenge as you'll need the latest version of Wine and at the time these instructions where written none of the CentOS repositories contained pre-built rpms for it.

  1. Log into the system
  2. Upgrade your system if it hasn't already been
    sudo yum -y upgrade
  3. Install necessary packages for building and running wine
    yum -y installgroup "Development Tools"
    yum -y installgroup "X Window System"
    yum -y install libgcc.i686 glibc.i686 glibc-devel.i686 zlib-devel.i686 libX11.i686 libX11-devel.i686 freetype.i686 freetype-devel.i686 freetype-demos.x86_64 freetype-devel.x86_64 libxml2.i686 libxml2-devel.i686 libxslt.i686 libxslt-devel.i686 libXcursor.i686 libXcursor-devel.i686 libpng.i686 libpng-devel.i686 libjpeg.i686 libjpeg-devel.i686 libtiff.i686 libtiff-devel.i686 openssl.i686 openssl-devel.i686 libXrender.i686 libXrender-devel.i686 cabextract
  4. Download the latest development version of wine and install
  5. Get winetricks
    wget http://winetricks.org/winetricks
    chmod +x winetricks
  6. Install prerequisites
    ./winetricks winxp vcrun2005 vcrun2008 vcrun2010 dotnet40
  7. Complete steps 8 thru end above
[edit]

What Works?

The following table shows the results of testing Wine/msconvert with the various different instrument formats available to us. For each format the instrument output was converted using the same version of msconvert installed on Windows 7 and Wine. The program readmzXML was then used to dump a summary of the resulting mzML file and the values compared between systems.

If you have any questions about the testing or have additional results to add please forward them to the spc-tools discussion mailing list.

Wine/msconvert Instrument Formats Tested
Vendor Model Acquisition Software File Type Result Notes
ABSciex (Applied Biosystems) QQQ Analyst .wiff file Fails Throws exception "The type initializer for 'Clearcore2.Data.WiffReader.WiffSampleRun'"
Agilent QQQ MassHunter 2.1 .d directory Success
Agilent QTOF MassHunter 2.1 .d directory Success
Bruker Apex, MicrOTOF unknown .baf file Success
Thermo LTQ Orbitrap XL Xcalibur 2.4 SP1 .raw directory Success
Waters unknown MassLynx 4.1 .raw directory Success
[edit]

How can I install windows applications to be shared by multiple users under wine?

Wine does not currently allow sharing its Windows root (or "prefix") between users, due to the risk of registry corruption from running multiple wineservers simultaneously ([bug #11112]). This means you typically would need to go through and install each set of Windows applications for each user. If you have a lot of users this could become burdensome, however you can create a single prefix and then copy it for each user. (The default wine prefix is ~/.wine). This saves having to run the same installation instructions for each user.

Another option is to write a wrapper script around the whole process. This wrapper script could copy a standardized wine prefix with all of the applications installed into a new temporary prefix owned by the user, set the environment variable WINEPREFIX to point to the temp prefix, run the wine command(s), stop the user's wine server, and then remove the temporary prefix.

One last item, be careful about using NFS with wine. If your home directory is mounted using NFS and shared across systems you could experience some problems running wine on multiple systems in parallel. More information about wine's "windows" partition and NFS:

http://www.winehq.org/pipermail/wine-users/2010-July/076137.html
[edit]

References

http://groups.google.com/group/spctools-discuss/browse_thread/thread/604745729ac0e541
http://groups.google.com/group/spctools-discuss/browse_thread/thread/bb2a608ad31f77dd/868fab027488bd09?hl=en&lnk=gst&q=wine#868fab027488bd09
http://groups.google.com/group/spctools-discuss/browse_thread/thread/2d98145b42479cb0/f8d80a25d1f8583c?hl=en&lnk=gst&q=wine#f8d80a25d1f8583c
https://groups.google.com/d/msg/spctools-discuss/syqu3YBzzlg/tnSQIkCKhMQJ
Retrieved from "http://tools.proteomecenter.org/wiki/index.php?title=Msconvert_Wine"
Personal tools