oswitch 0.2.6 → 0.2.7

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 75af23e839e1c6933df699ecb6e14f5512ce863c
-  data.tar.gz: 902745b152459f284ff363b6897efba8e2c341ab
+  metadata.gz: 9c4115b5b4f04909c25cb4dd04baa720e3a0384e
+  data.tar.gz: cd0859b22016d60e6d278d6b26ff24eb4643ffa3
 SHA512:
-  metadata.gz: 9ee1019f702540123b5bcf4305059eb432d00946564f820770d2aeec4dbfc0d220cff2b927f16ab47eddeb9bbdcaf8c69a18c4fea659abaa230ff297dae153d2
-  data.tar.gz: 6c08c54c9bc36fcaf58d2aab8814da44b6ef50b7b6dc41cee1dc6a21a0d22bf42dceb37c9af6540106bcc3c65fc06a28e5c605a275b4710cae7d7c510eacf12b
+  metadata.gz: 618de17e8315c607b14aa43bb804cbaae61bb8726ec9b3833fdaed0a134852d62bf19da2227e0015ae225d954333b296500c45ba6c32ccf68888b91c8c0ea4a0
+  data.tar.gz: 6e191e6ee60604e1b9111cc31b0fa93409fb52ba2f52077fb5a4d8b4b50a89243a349a3d147d5daa6caf95b6a03fa6e3e69e03493751cec20f0205394bcc5459

data/README.mkd

@@ -1,18 +1,22 @@
-# OSwitch
+# oswitch - virtual environments for flexible and reproducible bioinformatics analyses
 
-One-line access to other operating systems.
-
-Mini-example:
+More and more bioinformatics software are being "containerised" with [docker](http://docker.io/), making them installable on personal computers and compute clusters alike with a single command in a reproducible manner. However, using containerised software can still be a challenge as the containers are effectively a different operating system altogether.
+oswitch enhances usability of docker containers by a) automatically making available the local file-system and user profile inside the container, b) using the host user's login shell and current working directory as the entry point. The net effect is similar to entering "virtual environment" on the host system containing specific versions of software of interest.
 
 ```bash
-mymacbook:~/2015-02-01-myproject> abyss-pe
+mymacbook:~/2015-02-01-myproject> abyss-pe k=25 reads.fastq.gz
     zsh: command not found: abyss-pe
+
+# List available images.
 mymacbook:~/2015-02-01-myproject> oswitch -l
     yeban/biolinux:8
     ubuntu:14.04
     ontouchstart/texlive-full
     ipython/ipython
     hlapp/rpopgen
+    bioconductor/release_sequencing
+
+# Enter the continaer and run commands interactively.
 mymacbook:~/2015-02-01-myproject> oswitch biolinux
     ###### You are now running: biolinux in container biolinux-7187. ######
 biolinux-7187:~/2015-02-01-myproject> abyss-pe k=25 reads.fastq.gz
@@ -20,258 +24,48 @@ biolinux-7187:~/2015-02-01-myproject> abyss-pe k=25 reads.fastq.gz
 biolinux-7187:~/2015-02-01-myproject> exit
 mymacbook:~/2015-02-01-myproject>
     [... output is where you expect it to be ...]
-```
 
-Detailed information:
-* [Features](#features)
-* [Usage](#example-usage)
-* [Installation](#installation)
-* [FAQ](#faq)
-* [Roadmap](#roadmap)
-* [Contributors & Funding](#contributors-&-funding)
-
-## Background
-
-Genomic analyses require jumping back and forth between many bioinformatics tools.
-The data types are young, thus so are the tools. This leads to frequent
-updates of tools that are often challenging to install. Furthermore, it is
-challenging to keep different versions of software for different projects, yet
-changing versions can make analyses difficult to reproduce. To make matters worse,
-genomicists often lack the skills necessary to setup complex bioinformatics software,
-and systems administrators can be overwhelmed by large numbers of software
-installation requests.
-
-## Aim
-
-We are developing `oswitch` to enable **seamless switching
-from one operating system to another** - providing access to diverse
-ranges of tools. This project grew from our own need to rapidly access 
-diverse pieces of specific versions of software including those distributed as part of
-[BioLinux](http://environmentalomics.org/bio-linux/) on our MacBooks
-and our university HPC system. This was previously too difficult, but is
-important for [reproducible research](http://www.software.ac.uk) and agility.
-@bmpvieira made it clear early on that the [docker](http://docker.io/) could
-help make something happen. We first shared the resulting concept during a
-[Balti & Bioinformatics](http://nickloman.github.io/balti/2014/05/01/balti-and-bioinformatics-27th-may-2014/)
-presentation in the form of this (now out of date!) slide:
-
-[![Slide from Balti](http://i.imgur.com/exfDi6a.png)](http://www.slideshare.net/yannickwurm/2014-0527-opinion-computing-for-genomics-sucks)
-
-Docker images feel similar to using virtual machine images - but are much more
-[flexible and light-weight](http://stackoverflow.com/questions/16047306/how-is-docker-io-different-from-a-normal-virtual-machine).
-They are thus easily [shared or published](https://hub.docker.com/). We are extremely lucky to
-be able to build upon this amazing technology.
-
-## Features
-
-**`oswitch` is a wrapper facilitating access to docker images**. Importantly, when switching
-operating systems inside a shell, most things remain unchanged:
-
-* Current working directory is maintained
-* User name, uid and gid are maintained
-* Login shell (bash/zsh/fish) is maintained
-* Home directory is maintained (thus all .dotfiles and config files
-  are maintained).
-* read/write permissions are maintained
-* Paths are maintained whenever possible. Thus volumes (external drives,
-NAS, USB) mounted on the host are available in the container at the same
-path.
-
-## Example Usage
-
-There are two broad usage scenarios: interactive use & non-interactive
-use.
-
-##### Use a package interactively in a normal command-line
-
-Minimalist example:
-
-```shell
-Yannick@n56-169 ~/myproject> uname -a
-Darwin n56-169.sbcs.qmul.ac.uk 14.0.0 Darwin Kernel Version 14.0.0: Fri Sep 19 00:26:44 PDT 2014; root:xnu-2782.1.97~2/RELEASE_X86_64 x86_64
-Yannick@n56-169 ~/myproject> oswitch yeban/biolinux
-### You are now running: biolinux_8, in container: biolinux_8-27182. ###
-Yannick@biolinux_8-27182 ~/myproject> uname -a
-Linux biolinux_8-27182 3.16.4-tinycore64 #1 SMP Thu Oct 23 16:14:24 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
-```
 
-Biologically relevant example:
-
-```shell
-# Trying to run blast.
-pixel:~/test/ $ ls 
-mygene.fasta
-pixel:~/test/ $ cat mygene.fa
->myfavoritegene isthisone
-MNTLWLSLWDYPGKLPLNFMVFDTKDDLQAAYWRDPYSIPLAVIFEDPQPISQRLIYEIR
-TNPSYTLPPPPTKLYSAPISCRKNKTGHWMDDILSIKTGESCPVNNYLHSGFLALQMITD
-ITKIKLENSDVTIPDIKLIMFPKEPYTADWMLAFRVVIPLYMVLALSQFITYLLILIVGE
-KENKIKEGMKMMGLNDSVF
-pixel:~/test/ $ blastp -query mygene.fa -remote -db nr -outfmt 7 > mygene_blastp_nr.tab
-zsh: command not found: blastp
-# Indeed... blastp is missing from my MacBook. 
-
-# Switch to BioLinux and run blastp.
-pixel:~/test/ $ oswitch yeban/biolinux
-###### You are now running: biolinux in container biolinux-7187. ######
-biolinux-7187:~/test/ $ blastp -query mygene.fa -remote -db nr -outfmt 7 >  mygene_blastp_nr.tab
-# BioLinux includes blastp, thus the command ran smoothly.
-
-# View the result.
-biolinux-7187:~/test/ $ head mygene_blastp_nr.tab
-# BLASTP 2.2.28+
-# Query: myfavoritegene isthisone
-# RID: BJAHAHU9015
-# Database: nr
-# Fields: query id, subject id, % identity, alignment length, mismatches, gap opens, q. start, q. end, s. start, s. end, evalue, bit score
-# 501 hits found
-myfavoritegene	gi|322796550|gb|EFZ19024.1|	100.00	199	0	0	1	199	1	199	2e-142	 407
-myfavoritegene	gi|307183032|gb|EFN69988.1|	86.07	201	25	2	1	199	80	279	6e-115	 361
-myfavoritegene	gi|572260155|ref|XP_006608402.1|	80.60	201	36	2	1	199	95	294	4e-108	 350
-myfavoritegene	gi|328778864|ref|XP_397465.4|	80.60	201	36	2	1	199	95	294	5e-108	 350
-
-
-# [... potentially run other analyses that require biolinux things...]
-
-# Return to normal operating system
-biolinux-7187:~/test/ $ exit
-pixel:~/test/ $ ls 
-mygene.fasta mygene_blastp_nr.txt
-# our newly generated file is where we'd expect it to be.
-```
-
-##### Use a package non-interactively
-
-Alternatively, single commands can be run directly in a container
-(e.g. BioLinux) without entering it interactively. This can
-be useful to test new tools, or to run a single piece of
-not-locally-installed software as part of a single command. The
-container terminates automatically once the command has been
-executed, output is printed to the terminal and can be redirected, and
-the exit status of the command run within container is returned.
-
-```shell
-# Run command directly in BioLinux and view results if success.
+# Use a container non-interactively.
 pixel:~/test/ $ oswitch yeban/biolinux blastp -remote -query mygene.fa -db nr > mygene_blastp_nr.txt
 ```
 
-##### Listing available operating system containers
-
-OSwitch can pull any image from docker hub. You can see the images you pulled
-from docker hub using oswitch as:
-
-```shell
-pixel:~ $ oswitch -l
-yeban/biolinux:8
-ubuntu:14.04
-ontouchstart/texlive-full
-ipython/ipython
-hlapp/rpopgen
-```
-	
-##### Availability
-
-We have tested OSwitch on:
-
-* Mac OS X Yosemite
-* Ubuntu 14.04.1
-* CentOS 7
-
-##### Caveats
-
-* Some features work only for Debian, Ubuntu, CentOS based docker images.
-* Host directories/volumes with paths conflicting with container paths are
-  skipped.
-* SELinux must be disabled on CentOS for mounting volumes to work.
-* [Volume mounting on Mac OS hosts is imperfect](#q-directories-mounted-within-container-on-mac-host-are-empty).
-
 ## Installation
 
-OSwitch first requires a [working docker install](#Install and setup docker). 
-
-#### Install oswitch
-
-##### Mac
-
-Using [homebrew](http://brew.sh/).
-
-    $ brew install https://raw.githubusercontent.com/yeban/oswitch/master/homebrew/oswitch.rb
-
-Depending on whether Homebrew is installed systemwide or only for your user,
-this will install `oswitch` systemwide or only for your user.
-
-##### Linux
-
-###### Debian / Ubuntu
+oswitch first requires a [working docker install](https://github.com/wurmlab/Dockerfiles).
 
+#### On Ubuntu
 A `deb` package of `oswitch` is available in BioLinux repository for Trusty,
-Vivid and Jessie.
+Vivid and Jessie. This will install oswitch systemwide:
 
-    $ sudo add-apt-repository ppa:nebc/biolinux
+    $ sudo add-apt-repository ppa:nebc/bio-linux
     $ sudo apt-get update
     $ sudo apt-get install oswitch
 
-###### Others
-
-Requirements: Ruby 2.0 or higher.
-
-    $ gem install oswitch
-
-Depending on whether Ruby is installed systemwide (via your package-manager) or
-only for your user, this will install `oswitch` systemwide or only for your
-user.
+#### Using [homebrew](http://brew.sh/) (on Mac)
+Depending on whether homebrew is installed systemwide or only for your user,
+this will install oswitch systemwide or only for your user:
 
-#### Test oswitch
+    brew tap homebrew/science
+    brew install oswitch
 
-    $ oswitch ubuntu:14.04
+#### Using RubyGems (on Linux & Mac)
 
-## Install and setup docker
+Requirements: Ruby 2.0 or higher (available by default on Mac and through
+package managers on Linux). This will install oswitch systemwide:
 
-##### Mac OS X
+    $ sudo gem install oswitch
 
-Installing docker is much easier than before - https://docs.docker.com/installation/mac/
-
-##### Ubuntu
-
-Installing docker - https://docs.docker.com/installation/ubuntulinux/
-
-Add yourself to docker group so you can run docker client without sudo:
-
-```shell
-    $ sudo usermod -aG docker `whoami`
-    
-    # then logout and login again for the above command to take effect
-```
-	
-##### CentOS
+## Usage note
 
-Installing docker - https://docs.docker.com/installation/centos/
-
-Add yourself to docker group so you can run docker client without sudo:
-
-```shell
-    $ sudo usermod -aG docker `whoami`
-    
-    # then logout and login again for the above command to take effect
-```
-	
-Disable SELinux as it gets in the way of mounting volumes within the container:
-
-```shell
-    $ sed -i .bak 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
-
-    # then reboot your system
-```
-	
-The above command backs up the original file to `/etc/selinux/config.bak`. If
-you are concerned about disabling SELinux, do note that we are trying to work
-out a better solution.
-
-#### Test that docker is correctly installed
-
-The following should give an encouraging message: 
-
-    $ docker run hello-world
+* [Volume mounting on Mac OS hosts is imperfect](#q-directories-mounted-within-container-on-mac-host-are-empty).
+* SELinux must be disabled on CentOS hosts for mounting volumes to work (check
+  the SELinux documentation to see the implications of doing this).
+* We have tested oswitch on Debian, Ubuntu, CentOS based docker images on the
+  following hosts:
+  * Mac OS X Yosemite, El Captain
+  * Ubuntu 14.04.1
+  * CentOS 7
 
 ## FAQ
 
@@ -316,19 +110,8 @@ We create a new image on the fly that inherits from the given image. While creat
 the new image we execute a shell script that installs packages required for
 oswitch to work and creates a user in the image (almost) identical to that on the host.
 
-## Roadmap
-
-1. ~~make it possible to use docker containers without inheriting our
-current baseimage~~
-2. ~~gem distribution for easier installation~~
-3. ~~brew recipe for Mac~~
-4. ~~deb package~~
-5. test on QMUL's compute cluster
-6. make available images for common bioinformatics software
-7. deploy at [RAL/JASMIN](http://www.jasmin.ac.uk)
-8. create an SELinux policy to run oswitch on CentOS without having to disable
-SELinux entirely
-9. rpm package
+##### Q. How can I connect to an existing container?
+In another shell, use `docker ps` to see which containers are already running. Copy the identifier from the `CONTAINER ID` (column this looks something like `37e4e6ada6a4`), and use it to run `docker attach 37e4e6ada6a4` (replace with your container's id). This will create a new ssh connection to your existing container.
 
 ## Contribute
 

data/bin/oswitch

@@ -47,6 +47,13 @@ if options[:list]
   exit
 end
 
-package = ARGV[0]
-command = ARGV[1..-1]
-OSwitch.to(package, command)
+pattern = ARGV[0].strip
+command = ARGV[1..-1].join(' ')
+matches = OSwitch.packages.grep(/#{pattern}/)
+
+if matches.length > 1
+  puts "Which one of the following did you mean?"
+  puts matches.join(', ')
+else
+  OSwitch.to(matches.first ||  pattern, command)
+end

data/lib/oswitch.rb

@@ -40,8 +40,8 @@ class OSwitch
   end
 
   def initialize(package, command = [])
-    @package = package.strip
-    @command = command.join(' ')
+    @package = package
+    @command = command
     @imgname = "oswitch_#{@package}"
     @cntname = "#{@package.gsub(%r{/|:}, '_')}-#{Process.pid}"
     exec
@@ -51,9 +51,19 @@ class OSwitch
 
   def exec
     ping and build and switch
-  rescue ENODKR, ENOPKG => e
+  rescue ENODKR => e
     puts e
     exit
+  rescue => e
+    puts <<MSG
+
+Ouch! Looks like you have hit a bug. Please could you post the error report
+below to our issue tracker (https://github.com/wurmlab/oswitch/issues):
+
+#{e}\n#{e.backtrace.join("\n")}
+
+MSG
+    exit
   end
 
   private

data/lib/oswitch/exceptions.rb

@@ -1,14 +1,4 @@
 class OSwitch
-  class ENOPKG < StandardError
-    def initialize(name)
-      @name = name
-    end
-
-    def to_s
-      "Recipe to run #@name not available."
-    end
-  end
-
   class ENODKR < StandardError
     def to_s
       "***** Docker not installed / correctly setup / running.

data/lib/oswitch/os.rb

@@ -1,10 +1,23 @@
+require 'English'
+
 class OSwitch
   # Get OS specific info. Like, what directories to mount in the container,
-  # current user, home directory.
+  # current user, home directory, etc.
   #
   # This module first defines methods common to Linux and Darwin, then does
   # OS detection and loads OS specific code.
   module OS
+    # Return `true` if the given command exists and is executable.
+    def self.command?(command)
+      system("which #{command} > /dev/null 2>&1")
+    end
+
+    def self.outputof(command)
+      output = `#{command} 2> /dev/null`.chomp
+      raise unless $CHILD_STATUS.success?
+      output
+    end
+
     def username
       ENV['USER']
     end
@@ -21,13 +34,13 @@ class OSwitch
       Dir.pwd
     end
 
-    # Detect Linux or Darwin and load OS specific code. Following methods are
-    # added:
+    # Detect Linux or Darwin and load OS specific code. Following
+    # methods are added:
     #
     #   uid, gid, mountpoints
     #
     # NOTE:
-    #   This won't work on JRuby, as it sets RUBY_PLATFORM to 'java'.
+    # This won't work on JRuby as it sets RUBY_PLATFORM to 'java'.
     case RUBY_PLATFORM
     when /linux/
       require_relative 'os/linux'

data/lib/oswitch/os/darwin.rb

@@ -11,20 +11,34 @@ class OSwitch
           usr|var|Volumes$)
         }x
 
-      def uid
-        `boot2docker ssh id -u`.chomp
-      end
-
-      def gid
-        `boot2docker ssh id -g`.chomp
-      end
-
       def mountpoints
         volumes = Dir['/Volumes/*'].map {|v| File.symlink?(v) ? File.readlink(v) : v}
         volumes = volumes | Dir['/*']
         volumes.reject! { |mount| mount =~ BLACKLIST }
         volumes << home
       end
+
+      if OS.command?('docker-machine')
+        def uid
+          OS.outputof('docker-machine ssh default id -u')
+        end
+
+        def gid
+          OS.outputof('docker-machine ssh default id -g')
+        end
+      else
+        def uid
+        puts <<WARN
+'boot2docker' has been deprecated in favour of 'docker-machine'. Please upgrade
+via the Docker Toolbox (https://docker.com/toolbox).
+WARN
+          OS.outputof('boot2docker ssh id -u')
+        end
+
+        def gid
+          OS.outputof('boot2docker ssh id -g')
+        end
+      end
     end
   end
 end

data/oswitch.gemspec

@@ -1,7 +1,7 @@
 Gem::Specification.new do |s|
   # meta
   s.name        = 'oswitch'
-  s.version     = '0.2.6'
+  s.version     = '0.2.7'
   s.authors     = ['Anurag Priyam', 'Bruno Vieira', 'Yannick Wurm']
   s.email       = ['anurag08priyam@gmail.com']
   s.homepage    = 'https://github.com/yeban/oswitch'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: oswitch
 version: !ruby/object:Gem::Version
-  version: 0.2.6
+  version: 0.2.7
 platform: ruby
 authors:
 - Anurag Priyam
@@ -10,7 +10,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2015-09-15 00:00:00.000000000 Z
+date: 2017-06-25 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: colorize
@@ -70,10 +70,9 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.4.2
+rubygems_version: 2.5.2
 signing_key: 
 specification_version: 4
 summary: Use docker image as the host operating system's user and access host operating
   system's filesystem.
 test_files: []
-has_rdoc: