bio-samtools 0.6.0 → 0.6.2

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: a3abfc67e205b1963c0df6f664bace9fa5f8a890
+  data.tar.gz: cfc0474c9c8270a014b34e0125b36df2fc85ab02
+SHA512:
+  metadata.gz: bd34cefd2e15ad651382ade949f5e4a4e2e122958753fc18a3b245f629b3587f12e30c2af35a381bffafc89542a9e64679c80bd10122295f0ccfd6fa70570d96
+  data.tar.gz: 331edd40752b75dbdc3924cf16966f93c70b43617fbda5edaa4e98359f797e0c9234a016d93783ec90573c6aed470d850a332a945b7d94212963ca1332e743ab

data/.travis.yml

@@ -9,3 +9,5 @@ before_script:
   - cd ../
 rvm:
   - 1.9.3
+  - 2.0.0
+  - 2.1.0

data/Gemfile

@@ -4,15 +4,17 @@ source "http://rubygems.org"
 #   gem "activesupport", ">= 2.3.5"
 gem "ffi"
 gem "bio", ">= 1.4.2"
+gem "systemu", ">=2.5.2"
 
 # Add dependencies to develop your gem here.
 # Include everything needed to run rake, tests, features, etc.
 group :development do
   gem "shoulda", ">= 0"
-  gem "bundler", "~> 1.0.0"
+  gem "shoulda-context"
+  gem  "shoulda-matchers"
+  gem "bundler", "> 1.0.21"
   gem "jeweler"
-  gem "rcov", ">= 0"
-  gem "bio", ">= 1.4.2"
-  gem "ffi"
+  gem "rcov", ">=0", :platforms => :ruby_18
+  gem "simplecov", ">= 0",   :platforms => :ruby_19
   gem 'rdoc'
 end

data/README.md

@@ -0,0 +1,112 @@
+# bio-samtools
+
+The original project samtools-ruby belongs to Ricardo H. Ramirez @ [https://github.com/homonecloco/samtools-ruby] (https://github.com/homonecloco/samtools-ruby)
+
+## Introduction
+
+Documentation and code come from that project and we'll adapt it for a better integration in BioRuby.
+
+Binder of samtools for ruby, on the top of FFI. 
+
+This project was born from the need to add support of BAM files to 
+the [gee_fu genome browser] (http://github.com/danmaclean/gee_fu). 
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+    gem 'bio-samtools'
+    
+And then execute:
+ 
+    bundle
+    
+Or install it yourself as:
+
+    $ gem install bio-samtools
+
+## Usage
+
+### Creating a new SAM object
+
+A SAM object represents the alignments in the BAM file, and is very straightforward to create, you will need a sorted BAM file, to access the alignments and a reference sequence in FASTA format to use the reference sequence. The object can be created and opened as follows:
+
+	require 'bio-samtools'
+	
+	bam = Bio::DB::Sam.new(:bam=>"my_sorted.bam", :fasta=>'ref.fasta')
+	bam.open
+
+### Getting Reference Sequence
+
+Retrieving the reference can only be done if the reference has been loaded, which isn't done automatically in order to save memory. Reference need only be loaded once, and is accessed using reference name, start, end in 1-based co-ordinates. A standard Ruby String object is returned.
+
+	bam.load_reference 
+	sequence_fragment = bam.fetch_reference("Chr1", 1, 500)
+	
+### Getting Alignments
+
+Alignments can be obtained one at a time by looping over a specified region using the fetch() function.
+
+	bam.load_reference 
+	bam.fetch("1",3000,4000).each do |alignment|
+		#do something with the alignment...
+	end
+
+See more detail on doc/tutorial.html or doc/tutorial.pdf for a walkthrough tutorial. 
+
+## Dependencies
+* FFI [http://github.com/ffi/ffi](http://github.com/ffi/ffi)
+* BioRuby >= 1.4.1 [https://github.com/bioruby/bioruby](https://github.com/bioruby/bioruby)
+
+	
+## FAQ
+* I´m getting a **segmentation Fault**, what did I do wrong?
+
+	[Answer] There are two known segmentation faults at the moment
+
+	* When you try to load a text file as binary file
+	* When you try to lad a binary file as a text file
+
+* I keep seeing this **Invalid gemspec in [some ruby gem path…]**, what is wrong?
+	
+	[Answer] This appears to be a bug in RubyGems that doesn't affect the running of the tools. It will keep happening until someone updates RubyGems. If it really bugs you, downgrade RubyGems.
+
+## Contributing to bio-samtools
+ 
+* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet
+* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it
+* Fork the project
+* Start a feature/bugfix branch
+* Commit and push until you are happy with your contribution
+* Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
+* Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
+
+### TODO
+1. Filter to the fetching algorithm (give a condition that has to be satisfied to add the alignment to the list)
+
+### To whom do I complain?
+Try Ricardo dot Ramirez-Gonzalez at tgac dot ac dot uk
+ and [dan.maclean@tsl.ac.uk](dan.maclean@tsl.ac.uk)
+
+### Important Notes
+* Libraries (libbam) are downloaded, compiled and installed inside the gem at install time on the host system
+
+    `openssl dgst libbam.so.1` MD5 is c45cfccfb41ffeb2730ee4b227d244c4
+
+### Important Notes for developers
+
+Remember that you must compile and install the right libbam library for you host system. In order to do that there are three possible solutions:
+
+* download, compile and install the library in bioruby-samtools-your_clone/lib/bio/db/sam/external/libbam.xxxxx by yourself
+* install the gem and then grab the compiled library `cp 'locate libbam.1.dylib' bioruby-samtools-your_clone/lib/bio/db/sam/external` (library name is an example)
+* in your bioruby-samtools-your_clone create the Rakefile typing `cd ext; ruby mkrf_conf.rb; rake -f Rakefile`
+
+The latest I think is the easiest way, cause you are replicating the automatic process.
+
+For testing just run `rake test`. Tests must be improved.
+
+## Copyright
+
+Copyright (c) 2011 Raoul J.P. Bonnal. See LICENSE.txt for
+further details.
+

data/Rakefile

@@ -39,12 +39,15 @@ Rake::TestTask.new(:test) do |test|
   test.verbose = true
 end
 
+
+if RUBY_VERSION.start_with?("1.8")
 require 'rcov/rcovtask'
 Rcov::RcovTask.new do |test|
   test.libs << 'test'
   test.pattern = 'test/**/test_*.rb'
   test.verbose = true
 end
+end
 
 task :default => :test
 

data/VERSION

@@ -1 +1 @@
-0.6.0
+0.6.2

data/bio-samtools.gemspec

@@ -2,28 +2,30 @@
 # DO NOT EDIT THIS FILE DIRECTLY
 # Instead, edit Jeweler::Tasks in Rakefile, and run 'rake gemspec'
 # -*- encoding: utf-8 -*-
+# stub: bio-samtools 0.6.2 ruby lib
+# stub: ext/mkrf_conf.rb
 
 Gem::Specification.new do |s|
   s.name = "bio-samtools"
-  s.version = "0.6.0"
+  s.version = "0.6.2"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
+  s.require_paths = ["lib"]
   s.authors = ["Ricardo Ramirez-Gonzalez", "Dan MacLean", "Raoul J.P. Bonnal"]
-  s.date = "2012-12-03"
+  s.date = "2014-02-25"
   s.description = "Binder of samtools for ruby, on the top of FFI. \n\n  This project was born from the need to add support of BAM files to \n  the gee_fu genome browser (http://github.com/danmaclean/gee_fu)."
   s.email = "ilpuccio.febo@gmail.com"
   s.extensions = ["ext/mkrf_conf.rb"]
   s.extra_rdoc_files = [
     "LICENSE.txt",
-    "README.rdoc"
+    "README.md"
   ]
   s.files = [
     ".document",
     ".travis.yml",
     "Gemfile",
-    "Gemfile.lock",
     "LICENSE.txt",
-    "README.rdoc",
+    "README.md",
     "Rakefile",
     "VERSION",
     "bio-samtools.gemspec",
@@ -154,6 +156,7 @@ Gem::Specification.new do |s|
     "doc/tutorial.html",
     "doc/tutorial.pdf",
     "ext/Makefile-bioruby.patch",
+    "ext/Makefile-suse.patch",
     "ext/mkrf_conf.rb",
     "lib/bio-samtools.rb",
     "lib/bio/.DS_Store",
@@ -167,6 +170,11 @@ Gem::Specification.new do |s|
     "lib/bio/db/sam/sam.rb",
     "lib/bio/db/vcf.rb",
     "test/helper.rb",
+    "test/samples/pipe_char/test.bam",
+    "test/samples/pipe_char/test.bam.bai",
+    "test/samples/pipe_char/test.tam",
+    "test/samples/pipe_char/test_chr.fasta",
+    "test/samples/pipe_char/test_chr.fasta.fai",
     "test/samples/small/ids2.txt",
     "test/samples/small/sorted.bam",
     "test/samples/small/sorted.bam.bai",
@@ -193,43 +201,48 @@ Gem::Specification.new do |s|
   ]
   s.homepage = "http://github.com/helios/bioruby-samtools"
   s.licenses = ["MIT"]
-  s.require_paths = ["lib"]
-  s.rubygems_version = "1.8.24"
+  s.rubygems_version = "2.2.1"
   s.summary = "Binder of samtools for ruby, on the top of FFI."
 
   if s.respond_to? :specification_version then
-    s.specification_version = 3
+    s.specification_version = 4
 
     if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
       s.add_runtime_dependency(%q<ffi>, [">= 0"])
       s.add_runtime_dependency(%q<bio>, [">= 1.4.2"])
+      s.add_runtime_dependency(%q<systemu>, [">= 2.5.2"])
       s.add_development_dependency(%q<shoulda>, [">= 0"])
-      s.add_development_dependency(%q<bundler>, ["~> 1.0.0"])
+      s.add_development_dependency(%q<shoulda-context>, [">= 0"])
+      s.add_development_dependency(%q<shoulda-matchers>, [">= 0"])
+      s.add_development_dependency(%q<bundler>, ["> 1.0.21"])
       s.add_development_dependency(%q<jeweler>, [">= 0"])
       s.add_development_dependency(%q<rcov>, [">= 0"])
-      s.add_development_dependency(%q<bio>, [">= 1.4.2"])
-      s.add_development_dependency(%q<ffi>, [">= 0"])
+      s.add_development_dependency(%q<simplecov>, [">= 0"])
       s.add_development_dependency(%q<rdoc>, [">= 0"])
     else
       s.add_dependency(%q<ffi>, [">= 0"])
       s.add_dependency(%q<bio>, [">= 1.4.2"])
+      s.add_dependency(%q<systemu>, [">= 2.5.2"])
       s.add_dependency(%q<shoulda>, [">= 0"])
-      s.add_dependency(%q<bundler>, ["~> 1.0.0"])
+      s.add_dependency(%q<shoulda-context>, [">= 0"])
+      s.add_dependency(%q<shoulda-matchers>, [">= 0"])
+      s.add_dependency(%q<bundler>, ["> 1.0.21"])
       s.add_dependency(%q<jeweler>, [">= 0"])
       s.add_dependency(%q<rcov>, [">= 0"])
-      s.add_dependency(%q<bio>, [">= 1.4.2"])
-      s.add_dependency(%q<ffi>, [">= 0"])
+      s.add_dependency(%q<simplecov>, [">= 0"])
       s.add_dependency(%q<rdoc>, [">= 0"])
     end
   else
     s.add_dependency(%q<ffi>, [">= 0"])
     s.add_dependency(%q<bio>, [">= 1.4.2"])
+    s.add_dependency(%q<systemu>, [">= 2.5.2"])
     s.add_dependency(%q<shoulda>, [">= 0"])
-    s.add_dependency(%q<bundler>, ["~> 1.0.0"])
+    s.add_dependency(%q<shoulda-context>, [">= 0"])
+    s.add_dependency(%q<shoulda-matchers>, [">= 0"])
+    s.add_dependency(%q<bundler>, ["> 1.0.21"])
     s.add_dependency(%q<jeweler>, [">= 0"])
     s.add_dependency(%q<rcov>, [">= 0"])
-    s.add_dependency(%q<bio>, [">= 1.4.2"])
-    s.add_dependency(%q<ffi>, [">= 0"])
+    s.add_dependency(%q<simplecov>, [">= 0"])
     s.add_dependency(%q<rdoc>, [">= 0"])
   end
 end

data/ext/Makefile-bioruby.patch

@@ -1,27 +1,12 @@
---- Makefile	2011-10-27 14:57:06.000000000 +0200
-+++ ../Makefile-bioruby	2011-10-27 14:57:41.000000000 +0200
-@@ -1,10 +1,11 @@
+--- Makefile.bk	2014-02-01 19:56:14.000000000 +0000
++++ Makefile	2014-02-01 20:47:30.000000000 +0000
+@@ -1,7 +1,7 @@
  CC=			gcc
--CFLAGS=		-g -Wall -O2 #-m64 #-arch ppc
-+CFLAGS=		-g -Wall -O2 -fPIC #-m64 #-fPIC #-arch ppc
- DFLAGS=		-D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -D_USE_KNETFILE -D_CURSES_LIB=1
+-CFLAGS=		-g -Wall -O2
++CFLAGS=		-g -Wall -O2 -fPIC 
+ #LDFLAGS=		-Wl,-rpath,\$$ORIGIN/../lib
+-DFLAGS=		-D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -D_USE_KNETFILE -D_CURSES_LIB=1
++DFLAGS=		-D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -D_USE_KNETFILE -D_CURSES_LIB=0
  KNETFILE_O=	knetfile.o
  LOBJS=		bgzf.o kstring.o bam_aux.o bam.o bam_import.o sam.o bam_index.o	\
  			bam_pileup.o bam_lpileup.o bam_md.o razf.o faidx.o bedidx.o \
--			$(KNETFILE_O) bam_sort.o sam_header.o bam_reheader.o kprobaln.o bam_cat.o
-+			$(KNETFILE_O) bam_sort.o sam_header.o bam_reheader.o kprobaln.o bam_cat.o bam_plcmd.o \
-+			bam2bcf.o bam2bcf_indel.o sample.o bcftools/bcf.o bcftools/bcfutils.o errmod.o bcftools/fet.o
- AOBJS=		bam_tview.o bam_plcmd.o sam_view.o	\
- 			bam_rmdup.o bam_rmdupse.o bam_mate.o bam_stat.o bam_color.o	\
- 			bamtk.o kaln.o bam2bcf.o bam2bcf_indel.o errmod.o sample.o \
-@@ -81,8 +82,8 @@
- dylib:
- 		@$(MAKE) cleanlocal; \
- 		case `uname` in \
--			Linux) $(MAKE) CFLAGS="$(CFLAGS) -fPIC" libbam.so.1-local;; \
--			Darwin) $(MAKE) CFLAGS="$(CFLAGS) -fPIC" libbam.1.dylib-local;; \
-+			Linux) $(MAKE) CFLAGS="$(CFLAGS)  -fPIC" libbam.so.1-local;; \
-+			Darwin) $(MAKE) CFLAGS="$(CFLAGS) -fPIC" libbam.1.dylib-local ;; \
- 			*) echo 'Unknown OS';; \
- 		esac
- 

data/ext/Makefile-suse.patch

@@ -0,0 +1,11 @@
+--- Makefile	2013-12-10 14:06:29.868639418 +0000
++++ Makefile.opensuse	2013-12-10 14:06:56.548222174 +0000
+@@ -13,7 +13,7 @@
+ INCLUDES=	-I.
+ SUBDIRS=	. bcftools misc
+ LIBPATH=
+-LIBCURSES=	-lcurses # -lXCurses
++LIBCURSES=	-lncurses # -lXCurses
+ 
+ .SUFFIXES:.c .o
+ 

data/ext/mkrf_conf.rb

@@ -37,6 +37,8 @@ task :compile do
   sh "tar xvfj #{SamToolsFile}"
   cd("samtools-#{Version}") do
     sh "patch < ../Makefile-bioruby.patch"
+    # This patch replace CURSES lib with NCURSES which it is the only one available in OpenSUSE
+    sh "patch < ../Makefile-suse.patch"
     case Config::CONFIG['host_os']
       when /linux/
         #sh "CFLAGS='-g -Wall -O2 -fPIC' make -e"

data/lib/bio/db/sam.rb

@@ -4,6 +4,7 @@ require 'bio/db/sam/faidx'
 require 'bio/db/sam/sam'
 #require 'bio/db/pileup'
 #require 'bio/db/vcf'
+require 'systemu'
 
 module LibC
   extend FFI::Library
@@ -16,6 +17,9 @@ module Bio
   class DB
     class Sam
       attr_reader :sam_file
+      attr_accessor :minumum_ratio_for_iup_consensus
+      attr_reader :cached_regions
+      @minumum_ratio_for_iup_consensus = 0.20
 
       # To make a new sam object. Initialize expects a hash optsa with the following elemets:
       # fasta:: The fasta file with the reference. (nil)
@@ -158,17 +162,10 @@ module Bio
       #This is a simple average coverage just calculated with the first and last
       #possition of the alignment, ignoring the gaps.
       def chromosome_coverage(chromosome, qstart, len)
-        #  reference = fetch_reference(chromosome, qstart,len)
-        #  len = reference.length if len > reference.length
-        #p qend.to_s + "-" + qstart.to_s + "framesize " + (qend - qstart).to_s
+        
         coverages = Array.new(len, 0)
 
         chr_cov_proc = Proc.new do |alignment|
-          #last = qstart + len
-          #first = qstart
-          #last = alignment.calend if last > alignment.calend
-          #first = alignment.pos if first < alignment.pos
-          # p first
           last = alignment.calend - qstart
           first = alignment.pos - qstart
           if last < first
@@ -320,14 +317,21 @@ module Bio
                 sam_opts << k #strptrs << FFI::MemoryPointer.from_string(k)
                 sam_opts << v.to_s  unless ["-R", "-B", "-E", "-6", "-A"].include?(k) #these are just flags so don't pass a value... strptrs << FFI::MemoryPointer.from_string(v.to_s)
               end
+              sam_exe = File.join(File.expand_path(File.dirname(__FILE__)),'sam','external','samtools')
               sam_opts = sam_opts + ['-f', @fasta_path, @sam]
-              sam_command = "#{File.join(File.expand_path(File.dirname(__FILE__)),'sam','external','samtools')} mpileup #{sam_opts.join(' ')} 2> /dev/null"
-
-              sam_pipe = IO.popen(sam_command)
-              while line = sam_pipe.gets
-                yield Bio::DB::Pileup.new(line)
+              
+              sam_opts_string = SystemUniversal.quote(*sam_opts)
+              cmdline = "#{sam_exe} mpileup #{sam_opts_string}"
+              status, stdout, stderr = systemu cmdline
+              
+              if status.exitstatus == 0
+                stdout.each_line do |line|
+                  yield Bio::DB::Pileup.new(line)
+                end
+              else
+                raise SAMException.new(), "Error running mpileup. Command line was '#{cmdline}'\nsamtools STDERR was:\n#{stderr}"
               end
-              sam_pipe.close
+
               #strptrs << FFI::MemoryPointer.from_string('-f')
               #strptrs << FFI::MemoryPointer.from_string(@fasta_path)
               #strptrs << FFI::MemoryPointer.from_string(@sam)

data/lib/bio/db/sam/external/VERSION

@@ -1 +1 @@
-0.1.18
+0.1.19

data/lib/bio/db/sam/library.rb

@@ -16,9 +16,9 @@ module Bio
             'dll'
           else
             case RUBY_DESCRIPTION
-            when /darwin.*java/
+            when /jruby.*darwin/
               '1.dylib'
-            when /linux.*java/
+            when /jruby.*linux/
             'so.1'
             end
           end

data/lib/bio/db/sam/sam.rb

@@ -78,7 +78,7 @@ module Bio
         attach_function :samwrite, [ :pointer, :pointer ], :int
         attach_function :sampileup, [ :pointer, :int, :bam_pileup_f, :pointer ], :int
         attach_function :samfaipath, [ :string ], :string
-        attach_function :bam_mpileup, [ :int, :pointer ], :int
+#        attach_function :bam_mpileup, [ :int, :pointer ], :int
         attach_function :bam_idxstats, [:int, :pointer ], :int
       end
     end

data/test/samples/pipe_char/test.tam

@@ -0,0 +1,10 @@
+@SQ	SN:gi|123|chr_1	LN:69930
+test_1	0	gi|123|chr_1	135	0	70M	*	0	0	TAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAAC	*	XT:A:R	NM:i:0	X0:i:45	XM:i:0	XO:i:0	XG:i:0	MD:Z:70
+test_2	0	gi|123|chr_1	265	0	70M	*	0	0	CCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTA	*	XT:A:R	NM:i:0	X0:i:45	XM:i:0	XO:i:0	XG:i:0	MD:Z:70
+reverse_2	16	gi|123|chr_1	43	0	70M	*	0	0	CCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTA	*	XT:A:R	NM:i:0	X0:i:45	XM:i:0	XO:i:0	XG:i:0	MD:Z:70
+test_1_SNP	0	gi|123|chr_1	57	0	70M	*	0	0	TAACCCTAACCCTAACCCTAACCCTAACGCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAAC	*	XT:A:R	NM:i:1	X0:i:45	XM:i:1	XO:i:0	XG:i:0	MD:Z:28C41
+test_1_DEL	0	gi|123|chr_1	159	0	27M1D42M	*	0	0	TAACCCTAACCCTAACCCTAACCCTAACCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAAC	*	XT:A:R	NM:i:1	X0:i:45	XM:i:0	XO:i:1	XG:i:1	MD:Z:27^C42
+test_1_INS	0	gi|123|chr_1	207	0	29M1I41M	*	0	0	TAACCCTAACCCTAACCCTAACCCTAACCNCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAAC	*	XT:A:R	NM:i:1	X0:i:45	XM:i:0	XO:i:1	XG:i:1	MD:Z:70
+reverse_2_SNP	16	gi|123|chr_1	25	0	70M	*	0	0	CCTAACCCTAACCCTAACCCTAACCCTAACCCTACCCCTAACCCTAACCCTAACCCTAACCCTAACCCTA	*	XT:A:R	NM:i:1	X0:i:45	XM:i:1	XO:i:0	XG:i:0	MD:Z:34A35
+reverse_2_DEL	16	gi|123|chr_1	247	0	33M1D36M	*	0	0	CCTAACCCTAACCCTAACCCTAACCCTAACCCTACCCTAACCCTAACCCTAACCCTAACCCTAACCCTA	*	XT:A:R	NM:i:1	X0:i:45	XM:i:0	XO:i:1	XG:i:1	MD:Z:33^A36
+reverse_2_ins	16	gi|123|chr_1	169	0	35M1I35M	*	0	0	CCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCCTAACCCTAACCCTAACCCTAACCCTAACCCTA	*	XT:A:R	NM:i:1	X0:i:45	XM:i:0	XO:i:1	XG:i:1	MD:Z:70