Msconvert Wine

From SPCTools

(Difference between revisions)
Jump to: navigation, search
Revision as of 18:04, 30 June 2011
JoeS (Talk | contribs)
(What Works?)
← Previous diff
Current revision
Edeutsch (Talk | contribs)

Line 1: Line 1:
 +'''Last updated: 2019-01-09'''
 +
 +'''WARNING: As of 2019-01, the ProteoWizard developers report that the best way to run the latest msconvert with vendor format conversion on Linux is to use the official Docker image at:
 +
 +https://hub.docker.com/r/chambm/pwiz-skyline-i-agree-to-the-vendor-licenses
 +
 +The instructions below are kept for historical purposes, but if you want to install Wine that works with ProteoWizard yourself, it is suggested that you first refer to the official container's Dockerfile:
 +
 +https://github.com/ProteoWizard/container/blob/master/Dockerfile
 +
 +'''
 +
 +'''WARNING 2: This content below has not been updated in a long time and is probably obsolete. Below is a well-documented page on something that did work once at a point in time, but ProteoWizard and its dependencies, the vendor libraries, Wine, and the underlying operating systems are all moving targets. Therefore, the success story related below will likely not work on more recent platforms without additional tricks, or may not work at all. This page should be viewed as a story about something that did work once, and could work again, but may be difficult to get working on your particular platform. Better to use the references above.'''
 +
Proteowizard's msconvert utility is a must have tool for converting mass spectrometry data from almost any proprietary data format into open standards formats (primary mzML and mzXML). While this tool runs on both Windows and Linux, the ability to convert many common instrument formats under Linux is limited due to the need to have vendor specific libraries (dlls) installed. Fortunately it is possible to run the Windows version of msconvert under Linux using [http://www.winehq.org Wine], a open source compatibility layer for Windows -- ''though your mileage may vary''. Proteowizard's msconvert utility is a must have tool for converting mass spectrometry data from almost any proprietary data format into open standards formats (primary mzML and mzXML). While this tool runs on both Windows and Linux, the ability to convert many common instrument formats under Linux is limited due to the need to have vendor specific libraries (dlls) installed. Fortunately it is possible to run the Windows version of msconvert under Linux using [http://www.winehq.org Wine], a open source compatibility layer for Windows -- ''though your mileage may vary''.
Line 5: Line 19:
== Requirements == == Requirements ==
-* You '''will''' need to use wine in 32bit emulation mode (basically a 32bit wine even if you are on a 64bit architecture). This is because .Net and Visual Studio libraries are required and these libraries are installed using a 32bit installer, even for the 64 bit versions of these libraries. Please also be aware that not all proprietary vendor libraries that are distributed with ProtoWizard are 64 bit hence the 64 bit version of msconvert cannot be used to convert most proprietary files.+* You '''will''' need to use wine in 32bit emulation mode (basically a 32bit wine even if you are on a 64bit architecture). This is because the .Net and Visual Studio libraries are required and these libraries are installed using a 32 bit installer, even for the 64 bit versions of these libraries. Please also be aware that not all proprietary vendor libraries that are distributed with ProtoWizard are 64 bit hence the 64 bit version of msconvert cannot be used to convert most proprietary files.
-== Instructions for Ubuntu ==+== Instructions for Ubuntu (Updated 1/9/2013) ==
-The following instructions where tested using ProteoWizard 2.1.2785 on a Ubuntu 11.10 'natty' Amazon Machine Image running Wine 1.2.2 (See http://cloud.ubuntu.com/ami/ for more AMI options).+The following instructions where developed on a Amazon EC2 instance using Wine 1.5.21 and ProteoWizard 2.1.4131 on the official Ubuntu 12.04 'precise' Amazon Machine Image (AMI).
{| style="border-collapse: separate; border-spacing: 0; border-width: 1px; border-style: solid; border-color: #000; padding: 4px" {| style="border-collapse: separate; border-spacing: 0; border-width: 1px; border-style: solid; border-color: #000; padding: 4px"
Line 17: Line 31:
# Connect to the system you want to install on. If using ssh use the -X option to ensure display forwarding is set since parts of the setup will prompt you with GUI dialogs asking you to accept licenses. # Connect to the system you want to install on. If using ssh use the -X option to ensure display forwarding is set since parts of the setup will prompt you with GUI dialogs asking you to accept licenses.
-#:ssh -X root@system+#:ssh -X ubuntu@<hostname>
-# Update/install rpms+# Install the latest 32-bit version of wine
-#:sudo apt-get update+#:sudo add-apt-repository ppa:ubuntu-wine/ppa
-#:sudo apt-get -y install wine cabextract+#:sudo apt-get -y update
-# Use ''winetricks'' to install required Windows resources+#:sudo apt-get -y install wine1.5-i386
-#:wget http://www.kegel.com/wine/winetricks+#:sudo apt-get -y install winetricks
-#:chmod 755 winetricks+# Ensure wine has the correct architecture and set the location of the Windows emulation
-#:./winetricks winxp+#:export WINEARCH=win32
-#:./winetricks vcrun2008+#:export WINEPREFIX=~/.wine {{pad|5em}}''Default. Choose your location''
-#:./winetricks dotnet35 {{pad|3em}}''Will install .Net 2.0 followed by 3.0 then 3.5, and the 3.5''+# Use ''winetricks'' to configure the Windows emulation
-# Download and install Proteowizard+#:winetricks -q winxp vcrun2005 win7 vcrun2008 vcrun2010
-#: Go to http://proteowizard.sourceforge.net/ using a web browser. Click on the downloads link and select the Windows with vendor support option (not the 64 bit version as the required vendor libraries aren't 64 bit). Download the file with the tar.bz2 extension. Extract the package and move it to wherever you want to install it:+# Progressively install the .Net frameworks. While winetricks is suppose to install any older frameworks I found it works better if I did it manually starting from the oldest. Also see note below about sites blocking downloads.
-#:+#: winetricks -q dotnet20 dotnet20sp1 dotnet20sp2
-#::mkdir pwiz; tar xvf pwiz*.tar.bz2 -C pwiz+#: winetricks -q dotnet30
 +# Continue installing the rest of the .Net frameworks
 +#: winetricks dotnet30sp1
 +#: winetricks dotnet35''May fail. Rerunning it a 2nd time worked.''
 +#: winetricks dotnet35sp1
 +#: winetricks dotnet40
 +# Download and install Proteowizard.
 +#: Go to http://proteowizard.sourceforge.net/ using a web browser. Click on the downloads link and select the Windows with vendor support option with the tar.bz2 extension. Extract the package and move it to wherever you want to install it.
 +#:: mkdir pwiz; tar xvf pwiz*.tar.bz2 -C pwiz
 +# Register dll (shouldn't be needed but...)
 +#: wine regsvr32 pwiz/MSFileReader.XRawfile2.dll
 +#: wine regsvr32 pwiz/CompassXtractMS.dll
 +# Run msconvert
 +#:wine pwiz/msconvert.exe
==== Issues/Problems ==== ==== Issues/Problems ====
-* If you are trying to install on an Amazon EC2 instance winetricks may fail when trying to download and install vcrun2008 from Microsoft. From what I can tell Microsoft is blocking access from the Amazon Cloud. Just manually download the file from a different system (tip, use the wget command suggested by winetricks) and copy it to the EC2 instance. +===== Winetricks fails to download a .Net installer =====
-* If during the installation of .Net 3.0 it seems to hang, look in your "system tray" for the installer icon. This will be the final finish dialog that will need to be clicked through to end the installation.+ 
-* Currently doesn't work for AB Sciex wiff files as there is a problem with the vendor libraries, wine, and possibly .Net 3.5. Trying to determine the exact nature of the problem but it doesn't look promising at the moment.+For certain .Net installers winetricks may fail when trying to download the packages. From what I can tell some download sites may be blocking access. Just follow the instructions provided by winetricks. If you happen to be installing on a Amazon EC2 instance you may need to download the files from a system outside the cloud and copy the files to the instance.
-* Lastly should be aware that using .Net 3.5 under wine is still experimental at this time and the installer may report that it failed. ProteoWizard elements that make use of this may not work correctly.+ 
 +===== dotnet20sp1 install completed but reported missing file =====
 + 
 +You may observe the following message while installing .Net 20 SP1:
 + 
 + dotnet20sp1 install completed, but installed file ... /c:/windows/assembly/NativeImages_v2.0.50727_32/indexb.dat not found­
 + 
 +This appears to be a known issue that can occur with dotnet20sp1 (see [http://appdb.winehq.org/objectManager.php?bShowAll=true&bIsQueue=false&bIsRejected=false&sClass=version&sTitle=&sReturnTo=&iId=10166]). According to the report its harmless and can be fixed by creating creating an empty file as so:
 +<br>
 +<br>{{pad|5em}}mkdir -p $WINEPREFIX/dosdevices/c:/windows/assembly/NativeImages_v2.0.50727_32
 +<br>{{pad|5em}}touch $WINEPREFIX/dosdevices/c:/windows/assembly/NativeImages_v2.0.50727_32/indexb.dat
 + 
 +===== .Net 3.0 installation window "disappears" =====
 + 
 +If during the installation of .Net 3.0 the installation dialog disappears or appears to hang, look in your "system tray" for the installer icon. This will be the final finish dialog that will need to be clicked through to end the installation.
 + 
 +===== AB Sciex wiff files throw an exception =====
 + 
 +When trying to convert AB Sciex wiff files the following error is outputted by msconvert/wine:
 + 
 + [WiffFileImpl::getSampleNames] The type initializer for 'Clearcore2.Data.WiffReader.WiffSampleRun' threw an exception.
 + 
 +At this time msconvert under the wine emulator can't be used to convert AB Sciex wiff files.
== Instructions for CentOS == == Instructions for CentOS ==
-The following instructions where tested using ProteoWizard 2.1.2785 on a CentOS 5.5 Amazon Machine Image (ami-68f80f01). CentOS presents a bigger challenge, as you'll need the latest version of Wine and at the time these instructions where written none of the CentOS repositories contained pre-built rpms for it.+The following instructions where tested using ProteoWizard on a CentOS 6.3 Amazon Machine Image (ami-f0ab24c0). CentOS presents a bigger challenge as you'll need the latest version of Wine and at the time these instructions where written none of the CentOS repositories contained pre-built rpms for it.
 +# Log into the system
# Upgrade your system if it hasn't already been # Upgrade your system if it hasn't already been
-#: rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm+#: sudo yum -y upgrade
-#: rpm -Uvh http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS/rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm+
-#: yum -y update+
# Install necessary packages for building and running wine # Install necessary packages for building and running wine
-#: yum -y install xauth cabextract fontconfig gcc44 rpm-build+#: yum -y installgroup "Development Tools"
-#: wget ftp://ftp.pbone.net/mirror/ftp.sourceforge.net/pub/sourceforge/h/project/hp/hphp/CentOS%205%2064bit/RPM/flex-2.5.35-7.x86_64.rpm+#: yum -y installgroup "X Window System"
-#: rpm -Uvh flex*+#: yum -y install libgcc.i686 glibc.i686 glibc-devel.i686 zlib-devel.i686 libX11.i686 libX11-devel.i686 freetype.i686 freetype-devel.i686 freetype-demos.x86_64 freetype-devel.x86_64 libxml2.i686 libxml2-devel.i686 libxslt.i686 libxslt-devel.i686 libXcursor.i686 libXcursor-devel.i686 libpng.i686 libpng-devel.i686 libjpeg.i686 libjpeg-devel.i686 libtiff.i686 libtiff-devel.i686 openssl.i686 openssl-devel.i686 libXrender.i686 libXrender-devel.i686 cabextract
-# Log out, then log back in and build wine (using -X)+# Download the latest development version of wine and install
-#: CC=/usr/bin/gcc44+# Get winetricks
-#: wget http://download.hostedgameservers.com/install-wine1.2.2.sh+#:wget http://winetricks.org/winetricks
-#: chmod 777 install-wine1.2.2.sh+#:chmod +x winetricks
-#: ./install-wine1.2.2.sh +# Install prerequisites
-#: ''...And wait for a really really long time...''+#:./winetricks winxp vcrun2005 vcrun2008 vcrun2010 dotnet40
-# Run Steps 3 on from the ubtuntu section+# Complete steps 8 thru end above
-# Register the dll's+
-#: cd pwiz {{pad|3em}}''Or whichever directory you installed Proteowizard in''+
-#: wine regsvr32 MSFileReader.XRawfile2.dll+
-#: wine regsvr32 CompassXtractMS.dll+
== What Works? == == What Works? ==
Line 68: Line 112:
If you have any questions about the testing or have additional results to add please forward them to the spc-tools discussion mailing list. If you have any questions about the testing or have additional results to add please forward them to the spc-tools discussion mailing list.
 +<br>
<br> <br>
{| border="1" cellpadding="2" {| border="1" cellpadding="2"
|+ Wine/msconvert Instrument Formats Tested |+ Wine/msconvert Instrument Formats Tested
-! Vendor !! Model !! Acquisition Software !! Output Type !! Result !! Notes+! Vendor !! Model !! Acquisition Software !! File Type !! Result !! Notes
|- |-
-| Applied Biosystems (ABSciex)|| QQQ || Analyst || .wiff file +| ABSciex (Applied Biosystems)|| QQQ || Analyst || .wiff file
-| <span style="color:red">Fails</span> || Requires .Net 3.5 which doesn't function under wine.+| <span style="color:red">Fails</span> || Throws exception "The type initializer for 'Clearcore2.Data.WiffReader.WiffSampleRun'"
|- |-
| Agilent || QQQ || MassHunter 2.1 || .d directory | Agilent || QQQ || MassHunter 2.1 || .d directory
Line 83: Line 128:
| Success || | Success ||
|- |-
-| Bruker || Apex, MicrOTOF)|| ?1 || .baf file +| Bruker || Apex, MicrOTOF|| ''unknown'' || .baf file
 +| Success ||
 +|-
 +| Thermo || LTQ Orbitrap XL || Xcalibur 2.4 SP1 || .raw directory
| Success || | Success ||
|- |-
-| Waters || ? || MassLynx 4.1 || .raw directory +| Waters || ''unknown'' || MassLynx 4.1 || .raw directory
| Success || | Success ||
|} |}
Line 106: Line 154:
: http://groups.google.com/group/spctools-discuss/browse_thread/thread/bb2a608ad31f77dd/868fab027488bd09?hl=en&lnk=gst&q=wine#868fab027488bd09 : http://groups.google.com/group/spctools-discuss/browse_thread/thread/bb2a608ad31f77dd/868fab027488bd09?hl=en&lnk=gst&q=wine#868fab027488bd09
: http://groups.google.com/group/spctools-discuss/browse_thread/thread/2d98145b42479cb0/f8d80a25d1f8583c?hl=en&lnk=gst&q=wine#f8d80a25d1f8583c : http://groups.google.com/group/spctools-discuss/browse_thread/thread/2d98145b42479cb0/f8d80a25d1f8583c?hl=en&lnk=gst&q=wine#f8d80a25d1f8583c
 +: https://groups.google.com/d/msg/spctools-discuss/syqu3YBzzlg/tnSQIkCKhMQJ

Current revision

Last updated: 2019-01-09

WARNING: As of 2019-01, the ProteoWizard developers report that the best way to run the latest msconvert with vendor format conversion on Linux is to use the official Docker image at:

https://hub.docker.com/r/chambm/pwiz-skyline-i-agree-to-the-vendor-licenses

The instructions below are kept for historical purposes, but if you want to install Wine that works with ProteoWizard yourself, it is suggested that you first refer to the official container's Dockerfile:

https://github.com/ProteoWizard/container/blob/master/Dockerfile

WARNING 2: This content below has not been updated in a long time and is probably obsolete. Below is a well-documented page on something that did work once at a point in time, but ProteoWizard and its dependencies, the vendor libraries, Wine, and the underlying operating systems are all moving targets. Therefore, the success story related below will likely not work on more recent platforms without additional tricks, or may not work at all. This page should be viewed as a story about something that did work once, and could work again, but may be difficult to get working on your particular platform. Better to use the references above.

Proteowizard's msconvert utility is a must have tool for converting mass spectrometry data from almost any proprietary data format into open standards formats (primary mzML and mzXML). While this tool runs on both Windows and Linux, the ability to convert many common instrument formats under Linux is limited due to the need to have vendor specific libraries (dlls) installed. Fortunately it is possible to run the Windows version of msconvert under Linux using Wine, a open source compatibility layer for Windows -- though your mileage may vary.

The following are instructions on how to install and setup Wine under a variety of Linux platforms. These instructions where largely derived from several spctools-discuss emails and a large thanks goes out to the users who posted them (see references below).

Contents

Requirements

  • You will need to use wine in 32bit emulation mode (basically a 32bit wine even if you are on a 64bit architecture). This is because the .Net and Visual Studio libraries are required and these libraries are installed using a 32 bit installer, even for the 64 bit versions of these libraries. Please also be aware that not all proprietary vendor libraries that are distributed with ProtoWizard are 64 bit hence the 64 bit version of msconvert cannot be used to convert most proprietary files.

Instructions for Ubuntu (Updated 1/9/2013)

The following instructions where developed on a Amazon EC2 instance using Wine 1.5.21 and ProteoWizard 2.1.4131 on the official Ubuntu 12.04 'precise' Amazon Machine Image (AMI).

middle
Please read the notes regarding potential issues following these instructions before executing.
  1. Connect to the system you want to install on. If using ssh use the -X option to ensure display forwarding is set since parts of the setup will prompt you with GUI dialogs asking you to accept licenses.
    ssh -X ubuntu@<hostname>
  2. Install the latest 32-bit version of wine
    sudo add-apt-repository ppa:ubuntu-wine/ppa
    sudo apt-get -y update
    sudo apt-get -y install wine1.5-i386
    sudo apt-get -y install winetricks
  3. Ensure wine has the correct architecture and set the location of the Windows emulation
    export WINEARCH=win32
    export WINEPREFIX=~/.wine  Default. Choose your location
  4. Use winetricks to configure the Windows emulation
    winetricks -q winxp vcrun2005 win7 vcrun2008 vcrun2010
  5. Progressively install the .Net frameworks. While winetricks is suppose to install any older frameworks I found it works better if I did it manually starting from the oldest. Also see note below about sites blocking downloads.
    winetricks -q dotnet20 dotnet20sp1 dotnet20sp2
    winetricks -q dotnet30
  6. Continue installing the rest of the .Net frameworks
    winetricks dotnet30sp1
    winetricks dotnet35May fail. Rerunning it a 2nd time worked.
    winetricks dotnet35sp1
    winetricks dotnet40
  7. Download and install Proteowizard.
    Go to http://proteowizard.sourceforge.net/ using a web browser. Click on the downloads link and select the Windows with vendor support option with the tar.bz2 extension. Extract the package and move it to wherever you want to install it.
    mkdir pwiz; tar xvf pwiz*.tar.bz2 -C pwiz
  8. Register dll (shouldn't be needed but...)
    wine regsvr32 pwiz/MSFileReader.XRawfile2.dll
    wine regsvr32 pwiz/CompassXtractMS.dll
  9. Run msconvert
    wine pwiz/msconvert.exe

Issues/Problems

Winetricks fails to download a .Net installer

For certain .Net installers winetricks may fail when trying to download the packages. From what I can tell some download sites may be blocking access. Just follow the instructions provided by winetricks. If you happen to be installing on a Amazon EC2 instance you may need to download the files from a system outside the cloud and copy the files to the instance.

dotnet20sp1 install completed but reported missing file

You may observe the following message while installing .Net 20 SP1:

dotnet20sp1 install completed, but installed file ... /c:/windows/assembly/NativeImages_v2.0.50727_32/indexb.dat not found­

This appears to be a known issue that can occur with dotnet20sp1 (see [1]). According to the report its harmless and can be fixed by creating creating an empty file as so:

 mkdir -p $WINEPREFIX/dosdevices/c:/windows/assembly/NativeImages_v2.0.50727_32
 touch $WINEPREFIX/dosdevices/c:/windows/assembly/NativeImages_v2.0.50727_32/indexb.dat

.Net 3.0 installation window "disappears"

If during the installation of .Net 3.0 the installation dialog disappears or appears to hang, look in your "system tray" for the installer icon. This will be the final finish dialog that will need to be clicked through to end the installation.

AB Sciex wiff files throw an exception

When trying to convert AB Sciex wiff files the following error is outputted by msconvert/wine:

[WiffFileImpl::getSampleNames] The type initializer for 'Clearcore2.Data.WiffReader.WiffSampleRun' threw an exception.

At this time msconvert under the wine emulator can't be used to convert AB Sciex wiff files.

Instructions for CentOS

The following instructions where tested using ProteoWizard on a CentOS 6.3 Amazon Machine Image (ami-f0ab24c0). CentOS presents a bigger challenge as you'll need the latest version of Wine and at the time these instructions where written none of the CentOS repositories contained pre-built rpms for it.

  1. Log into the system
  2. Upgrade your system if it hasn't already been
    sudo yum -y upgrade
  3. Install necessary packages for building and running wine
    yum -y installgroup "Development Tools"
    yum -y installgroup "X Window System"
    yum -y install libgcc.i686 glibc.i686 glibc-devel.i686 zlib-devel.i686 libX11.i686 libX11-devel.i686 freetype.i686 freetype-devel.i686 freetype-demos.x86_64 freetype-devel.x86_64 libxml2.i686 libxml2-devel.i686 libxslt.i686 libxslt-devel.i686 libXcursor.i686 libXcursor-devel.i686 libpng.i686 libpng-devel.i686 libjpeg.i686 libjpeg-devel.i686 libtiff.i686 libtiff-devel.i686 openssl.i686 openssl-devel.i686 libXrender.i686 libXrender-devel.i686 cabextract
  4. Download the latest development version of wine and install
  5. Get winetricks
    wget http://winetricks.org/winetricks
    chmod +x winetricks
  6. Install prerequisites
    ./winetricks winxp vcrun2005 vcrun2008 vcrun2010 dotnet40
  7. Complete steps 8 thru end above

What Works?

The following table shows the results of testing Wine/msconvert with the various different instrument formats available to us. For each format the instrument output was converted using the same version of msconvert installed on Windows 7 and Wine. The program readmzXML was then used to dump a summary of the resulting mzML file and the values compared between systems.

If you have any questions about the testing or have additional results to add please forward them to the spc-tools discussion mailing list.

Wine/msconvert Instrument Formats Tested
Vendor Model Acquisition Software File Type Result Notes
ABSciex (Applied Biosystems) QQQ Analyst .wiff file Fails Throws exception "The type initializer for 'Clearcore2.Data.WiffReader.WiffSampleRun'"
Agilent QQQ MassHunter 2.1 .d directory Success
Agilent QTOF MassHunter 2.1 .d directory Success
Bruker Apex, MicrOTOF unknown .baf file Success
Thermo LTQ Orbitrap XL Xcalibur 2.4 SP1 .raw directory Success
Waters unknown MassLynx 4.1 .raw directory Success

How can I install windows applications to be shared by multiple users under wine?

Wine does not currently allow sharing its Windows root (or "prefix") between users, due to the risk of registry corruption from running multiple wineservers simultaneously ([bug #11112]). This means you typically would need to go through and install each set of Windows applications for each user. If you have a lot of users this could become burdensome, however you can create a single prefix and then copy it for each user. (The default wine prefix is ~/.wine). This saves having to run the same installation instructions for each user.

Another option is to write a wrapper script around the whole process. This wrapper script could copy a standardized wine prefix with all of the applications installed into a new temporary prefix owned by the user, set the environment variable WINEPREFIX to point to the temp prefix, run the wine command(s), stop the user's wine server, and then remove the temporary prefix.

One last item, be careful about using NFS with wine. If your home directory is mounted using NFS and shared across systems you could experience some problems running wine on multiple systems in parallel. More information about wine's "windows" partition and NFS:

http://www.winehq.org/pipermail/wine-users/2010-July/076137.html

References

http://groups.google.com/group/spctools-discuss/browse_thread/thread/604745729ac0e541
http://groups.google.com/group/spctools-discuss/browse_thread/thread/bb2a608ad31f77dd/868fab027488bd09?hl=en&lnk=gst&q=wine#868fab027488bd09
http://groups.google.com/group/spctools-discuss/browse_thread/thread/2d98145b42479cb0/f8d80a25d1f8583c?hl=en&lnk=gst&q=wine#f8d80a25d1f8583c
https://groups.google.com/d/msg/spctools-discuss/syqu3YBzzlg/tnSQIkCKhMQJ
Personal tools