htslib 0.2.2 → 0.2.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 93ee916dafed0ba04e486a59c81d2bd9befbceda3d679ef820e32d8817597afe
-  data.tar.gz: 7dda13879d0c3707354ab69fe9967f60f0300c1971011218a5cf79e5e211ee25
+  metadata.gz: 2a49c4758453c5c39915d9d17ccc33ce104699fca39c4083d4b2801767b655f1
+  data.tar.gz: 46ee49f6b452471e26afedfce70489cd24eebf86b1a18240bab125d465921fb8
 SHA512:
-  metadata.gz: 8cf68e9276f353bdea4389f5584066870a4887f07891387cf143408cf215c43232f2c390c08b80c57225a8d8b3722a9dcc5ba5cc3cc716079cf004bd34df0da0
-  data.tar.gz: f30abf9c0d0dd07201e150dea5ec6976e7d968bc2f6bf2c347c1dec18cdfaaa5c141793208587f1629fa3cefeaaadd268083a105ab42dd450ce2ac8cfa0e2d7e
+  metadata.gz: 918c0919750eb3d436cf19b1011ffed0a8d0760b29e0a3c2e675a9efa563a3f5c3b9a9ae38232e67192df4d066c446bbc0443f00d07cab9ac0c3401b38bd3fb7
+  data.tar.gz: 343fbaec89a2e60a54feadd29856b26b772baa65ef52d181cd5845ed53a8e549504ddcdcc739dff716f9069a5b0b3e6f22f5136c2a612cc6014eb79ee225ab2b

data/README.md

@@ -8,9 +8,7 @@
 
 Ruby-htslib is the [Ruby](https://www.ruby-lang.org) bindings to [HTSlib](https://github.com/samtools/htslib), a C library for high-throughput sequencing data formats. It allows you to read and write file formats commonly used in genomics, such as [SAM, BAM, VCF, and BCF](http://samtools.github.io/hts-specs/), in the Ruby language.
 
-:apple: Feel free to fork it out if you can develop it! 
-
-:bowtie: Alpha stage.
+:apple: Feel free to fork it! 
 
 ## Requirements
 
@@ -18,6 +16,7 @@ Ruby-htslib is the [Ruby](https://www.ruby-lang.org) bindings to [HTSlib](https:
 * [HTSlib](https://github.com/samtools/htslib)
   * Ubuntu : `apt install libhts-dev`
   * macOS : `brew install htslib`
+  * Windows : [mingw-w64-htslib](https://packages.msys2.org/base/mingw-w64-htslib) is automatically fetched when installing the gem ([RubyInstaller](https://rubyinstaller.org) only).
   * Build from source code (see Development section)
 
 ## Installation
@@ -34,11 +33,13 @@ Alternatively, you can specify the directory of the shared library by setting th
 export HTSLIBDIR="/your/path/to/htslib" # libhts.so
 ```
 
+ruby-htslib also works on Windows; if you use RubyInstaller, htslib will be prepared automatically.
+
 ## Overview
 
-### High level API
+### High-level API
 
-Read SAM / BAM / CRAM - Sequence Alignment Map file
+HTS::Bam - SAM / BAM / CRAM - Sequence Alignment Map file
 
 ```ruby
 require 'htslib'
@@ -56,14 +57,14 @@ bam.each do |r|
      mpos: r.mpos + 1,
      isiz: r.isize,
      seqs: r.seq,
-     qual: r.qual.map { |i| (i + 33).chr }.join,
+     qual: r.qual_string,
      MC:   r.aux("MC")
 end
 
 bam.close
 ```
 
-Read VCF / BCF - Variant Call Format file
+HTS::Bcf - VCF / BCF - Variant Call Format file
 
 ```ruby
 bcf = HTS::Bcf.open("b.bcf")
@@ -96,9 +97,18 @@ fa.close
 
 </details>
 
-### Low level API
+<details>
+<summary><b>Tbx</b></summary>
+
+```ruby
+
+```
 
-`HTS::LibHTS` provides native C functions. 
+</details>
+
+### Low-level API
+
+`HTS::LibHTS` provides native C functions.
 
 ```ruby
 require 'htslib'
@@ -109,11 +119,17 @@ p b[:category]
 p b[:format]
 ```
 
-Note: htslib makes extensive use of macro functions for speed. You cannot use C macro functions in Ruby if they are not reimplemented in ruby-htslib. Only small number of C structs are implemented with FFI's ManagedStruct, which frees memory when Ruby's garbage collection fires. Other structs will need to be freed manually. 
+Macro functions
+
+htslib has a lot of macro functions for speed. Ruby-FFI cannot call C macro functions. However, essential functions are reimplemented in Ruby, and you can call them.
+
+Structs
+
+Only a small number of C structs are implemented with FFI's ManagedStruct, which frees memory when Ruby's garbage collection fires. Other structs will need to be freed manually.
 
 ### Need more speed?
 
-Try Crystal. [htslib.cr](https://github.com/bio-crystal/htslib.cr) is implemented in Crystal language and provides an API compatible with ruby-htslib. Crystal language is not as flexible as Ruby language. You can not use `eval` methods, and you must always be careful with the data types. Writing one-time scripts in Crystal or playing with REPL may not be as much fun. However, if you have a clear idea of what you want to do in your mind, have already written code in Ruby, and need to run them over and over, try creating a command line tool in Crystal.  The Crystal language is fast, as fast as the Rust and C languages. It will give you great power to create tools.
+Try Crystal. [HTS.cr](https://github.com/bio-cr/hts.cr) is implemented in Crystal language and provides an API compatible with ruby-htslib. Crystal language is not as flexible as Ruby language. You can not use `eval` methods and must always be careful with the types. Writing one-time scripts in Crystal may be less fun. However, if you have a clear idea of what you want to do in your mind, have already written code in Ruby, and need to run them over and over, try creating a command line tool in Crystal. The Crystal language is as fast as the Rust and C languages. It will give you incredible power to make tools.
 
 ## Documentation
 
@@ -134,26 +150,29 @@ bundle exec rake test
 
 [GNU Autotools](https://en.wikipedia.org/wiki/GNU_Autotools) is required to compile htslib.
 
-Many macro functions are used in HTSlib. Since these macro functions cannot be called using FFI, they must be reimplemented in Ruby.
+HTSlib has many macro functions. These macro functions cannot be called from FFI and must be reimplemented in Ruby.
 
 * Use the new version of Ruby to take full advantage of Ruby's potential.
-  * This is possible because we have a small number of users. What a deal!
-* Remain compatible with [htslib.cr](https://github.com/bio-crystal/htslib.cr).
-  * The most challenging part is the return value. In the Crystal language, methods are expected to return only one type. On the other hand, in the Ruby language, methods that return multiple classes are very common. For example, in the Crystal language, the compiler gets confused if the return value is one of six types: Int32, Int64, Float32, Float64, Nil, or String. In fact Crystal can do this. But the code gets a little messy. In Ruby, this is very common and doesn't cause any problems. 
+  * This is possible because we have a small number of users.
+* Remain compatible with [HTS.cr](https://github.com/bio-cr/hts.cr).
+  * The most challenging part is the return value. In the Crystal language, methods are expected to return only one type. On the other hand, in the Ruby language, methods that return multiple classes are very common. For example, in the Crystal language, the compiler gets confused if the return value is one of six types: Int32, Int64, Float32, Float64, Nil, or String. In fact Crystal allows you to do that. But the code gets a little messy. In Ruby, this is very common and doesn't cause any problems.
+  * Ruby and Crystal are languages that use garbage collection. However, the memory release policy for allocated C structures is slightly different: in Ruby-FFI, you can define a `self.release` method in `FFI::Struct`. This method is called when GC. So you don't have to worry about memory in high-level APIs like Bam::Record or Bcf::Record, etc. Crystal requires you to define a finalize method on each class. So you need to define it in Bam::Record or Bcf::Record.
 
-In the script directory, there are several tools to help implement ruby-htslib. These tools may be forked into independent repository in the future.
+Method naming generally follows the Rust-htslib API. 
 
 #### FFI Extensions
 
 * [ffi-bitfield](https://github.com/kojix2/ffi-bitfield) : Extension of Ruby-FFI to support bitfields.
 
-#### Automatic generation or automatic validation (Future plan)
+#### Automatic validation
+
+In the `script` directory, there are several tools to help implement ruby-htslib. Scripts using c2ffi can check the coverage of htslib functions in Ruby-htslib. They are useful when new versions of htslib are released.
 
-+ [c2ffi](https://github.com/rpav/c2ffi) is a tool to create JSON format metadata from C header files. It is planned to use c2ffi to automatically generate bindings or tests.
+* [c2ffi](https://github.com/rpav/c2ffi) is a tool to create JSON format metadata from C header files. 
 
 ## Contributing
 
-Ruby-htslib is a library under development, so even small improvements like typofix are welcome! Please feel free to send us your pull requests.
+Ruby-htslib is a library under development, so even minor improvements like typo fixes are welcome! Please feel free to send us your pull requests.
 
 * [Report bugs](https://github.com/kojix2/ruby-htslib/issues)
 * Fix bugs and [submit pull requests](https://github.com/kojix2/ruby-htslib/pulls)
@@ -165,14 +184,14 @@ Ruby-htslib is a library under development, so even small improvements like typo
 ```
 # Ownership and Commitment Rights
 
-Do you need commit rights to ruby-htslib repository?
+Do you need commit rights to the ruby-htslib repository?
 Do you want to get admin rights and take over the project?
 If so, please feel free to contact us @kojix2.
 ```
 
-#### Why do you implement htslib in a language like Ruby, which is not widely used in the bioinformatics?
+#### Why do you implement htslib in a language like Ruby, which is not widely used in bioinformatics?
 
-One of the greatest joys of using a minor language like Ruby in bioinformatics is that there is nothing stopping you from reinventing the wheel. Reinventing the wheel can be fun. But with languages like Python and R, where many bioinformatics masters work, there is no chance left for beginners to create htslib bindings. Bioinformatics file formats, libraries and tools are very complex and I don't know how to understand them. So I wanted to implement the HTSLib binding myself to better understand how the pioneers of bioinformatics felt when establishing the file format and how they created their tools. I hope one day we can work on bioinformatics using Ruby and Crystal languages, not to replace other languages such as Python and R, but to add new power and value to this advancing field.
+One of the greatest joys of using a minor language like Ruby in bioinformatics is that nothing stops you from reinventing the wheel. Reinventing the wheel can be fun. But with languages like Python and R, where many bioinformatics masters work, there is no chance for beginners to create htslib bindings. Bioinformatics file formats, libraries, and tools are very complex, and I need to learn how to understand them. So I started to implement the HTSLib binding myself to better understand how the pioneers of bioinformatics felt when establishing the file format and how they created their tools. I hope one day we can work on bioinformatics using Ruby and Crystal languages, not to replace other languages such as Python and R, but to add new power and value to this advancing field.
 
 ## Links
 
@@ -182,6 +201,7 @@ One of the greatest joys of using a minor language like Ruby in bioinformatics i
 ## Funding support
 
 This work was supported partially by [Ruby Association Grant 2020](https://www.ruby.or.jp/en/news/20201022).
+
 ## License
 
 [MIT License](https://opensource.org/licenses/MIT).

data/lib/hts/bam/{aux.rb → auxi.rb}

@@ -1,5 +1,11 @@
 # frozen_string_literal: true
 
+# Q. Why is the file name auxi.rb and not aux.rb?
+#
+# A. This is for compatibility with Windows.
+# In Windows, aux is a reserved word
+# You cannot create a file named aux.
+
 module HTS
   class Bam < Hts
     # Auxiliary record data

data/lib/hts/bam/header.rb

@@ -1,11 +1,26 @@
 # frozen_string_literal: true
 
+require_relative "header_record"
+
 module HTS
   class Bam < Hts
     # A class for working with alignment header.
     class Header
-      def initialize(hts_file)
-        @sam_hdr = LibHTS.sam_hdr_read(hts_file)
+      def self.parse(str)
+        new(LibHTS.sam_hdr_parse(str.size, str))
+      end
+
+      def initialize(arg = nil)
+        case arg
+        when LibHTS::HtsFile
+          @sam_hdr = LibHTS.sam_hdr_read(arg)
+        when LibHTS::SamHdr
+          @sam_hdr = arg
+        when nil
+          @sam_hdr = LibHTS.sam_hdr_init
+        else
+          raise TypeError, "Invalid argument"
+        end
       end
 
       def struct
@@ -17,6 +32,7 @@ module HTS
       end
 
       def target_count
+        # FIXME: sam_hdr_nref
         @sam_hdr[:n_targets]
       end
 
@@ -32,6 +48,41 @@ module HTS
         end
       end
 
+      # experimental
+      def add_lines(str)
+        LibHTS.sam_hdr_add_lines(@sam_hdr, str, 0)
+      end
+
+      # experimental
+      def add_line(type, *args)
+        args = args.flat_map { |arg| [:string, arg] }
+        LibHTS.sam_hdr_add_line(@sam_hdr, type, *args, :pointer, FFI::Pointer::NULL)
+      end
+
+      # experimental
+      def find_line(type, key, value)
+        ks = LibHTS::KString.new
+        r = LibHTS.sam_hdr_find_line_id(@sam_hdr, type, key, value, ks)
+        r == 0 ? ks[:s] : nil
+      end
+
+      # experimental
+      def find_line_at(type, pos)
+        ks = LibHTS::KString.new
+        r = LibHTS.sam_hdr_find_line_pos(@sam_hdr, type, pos, ks)
+        r == 0 ? ks[:s] : nil
+      end
+
+      # experimental
+      def remove_line(type, key, value)
+        LibHTS.sam_hdr_remove_line_id(@sam_hdr, type, key, value)
+      end
+
+      # experimental
+      def remove_line_at(type, pos)
+        LibHTS.sam_hdr_remove_line_pos(@sam_hdr, type, pos)
+      end
+
       def to_s
         LibHTS.sam_hdr_str(@sam_hdr)
       end

data/lib/hts/bam/header_record.rb

@@ -0,0 +1,11 @@
+# frozen_string_literal: true
+
+module HTS
+  class Bam < Hts
+    class HeaderRecord
+      def initialize
+        @bam_hrec
+      end
+    end
+  end
+end

data/lib/hts/bam/record.rb

@@ -2,7 +2,7 @@
 
 require_relative "flag"
 require_relative "cigar"
-require_relative "aux"
+require_relative "auxi"
 
 module HTS
   class Bam < Hts
@@ -17,6 +17,7 @@ module HTS
         @header = header
       end
 
+      # Return the FFI::Struct object.
       def struct
         @bam1
       end
@@ -25,17 +26,14 @@ module HTS
         @bam1.to_ptr
       end
 
-      # returns the query name.
+      # Get the read name. (a.k.a QNAME)
+      # @return [String] query template name
       def qname
         LibHTS.bam_get_qname(@bam1).read_string
       end
 
-      # Set (query) name.
-      # def qname=(name)
-      #   raise 'Not Implemented'
-      # end
-
-      # returns the tid of the record or -1 if not mapped.
+      # Get the chromosome ID of the alignment. -1 if not mapped.
+      # @return [Integer] chromosome ID
       def tid
         @bam1[:core][:tid]
       end
@@ -44,7 +42,8 @@ module HTS
         @bam1[:core][:tid] = tid
       end
 
-      # returns the tid of the mate or -1 if not mapped.
+      # Get the chromosome ID of the mate. -1 if not mapped.
+      # @return [Integer] chromosome ID
       def mtid
         @bam1[:core][:mtid]
       end
@@ -53,7 +52,8 @@ module HTS
         @bam1[:core][:mtid] = mtid
       end
 
-      # returns 0-based start position.
+      # Get the 0-based leftmost coordinate of the alignment.
+      # @return [Integer] 0-based leftmost coordinate
       def pos
         @bam1[:core][:pos]
       end
@@ -62,7 +62,8 @@ module HTS
         @bam1[:core][:pos] = pos
       end
 
-      # returns 0-based mate position
+      # Get the 0-based leftmost coordinate of the mate.
+      # @return [Integer] 0-based leftmost coordinate
       def mate_pos
         @bam1[:core][:mpos]
       end
@@ -74,6 +75,8 @@ module HTS
       alias mpos mate_pos
       alias mpos= mate_pos=
 
+      # Get the bin calculated by bam_reg2bin().
+      # @return [Integer] bin
       def bin
         @bam1[:core][:bin]
       end
@@ -82,12 +85,15 @@ module HTS
         @bam1[:core][:bin] = bin
       end
 
-      # returns end position of the read.
+      # Get the rightmost base position of the alignment on the reference genome.
+      # @return [Integer] 0-based rightmost coordinate
       def endpos
         LibHTS.bam_endpos @bam1
       end
 
-      # returns the chromosome or '' if not mapped.
+      # Get the reference sequence name of the alignment. (a.k.a RNAME)
+      # '' if not mapped.
+      # @return [String] reference sequence name
       def chrom
         return "" if tid == -1
 
@@ -96,7 +102,9 @@ module HTS
 
       alias contig chrom
 
-      # returns the chromosome of the mate or '' if not mapped.
+      # Get the reference sequence name of the mate.
+      # '' if not mapped.
+      # @return [String] reference sequence name
       def mate_chrom
         return "" if mtid == -1
 
@@ -105,12 +113,20 @@ module HTS
 
       alias mate_contig mate_chrom
 
-      # Get strand information.
+      # Get whether the query is on the reverse strand.
+      # @return [String] strand "+" or "-"
       def strand
         LibHTS.bam_is_rev(@bam1) ? "-" : "+"
       end
 
-      # insert size
+      # Get whether the query's mate is on the reverse strand.
+      # @return [String] strand "+" or "-"
+      def mate_strand
+        LibHTS.bam_is_mrev(@bam1) ? "-" : "+"
+      end
+
+      # Get the observed template length. (a.k.a TLEN)
+      # @return [Integer] isize
       def insert_size
         @bam1[:core][:isize]
       end
@@ -122,7 +138,8 @@ module HTS
       alias isize insert_size
       alias isize= insert_size=
 
-      # mapping quality
+      # Get the mapping quality of the alignment. (a.k.a MAPQ)
+      # @return [Integer] mapping quality
       def mapq
         @bam1[:core][:qual]
       end
@@ -131,11 +148,14 @@ module HTS
         @bam1[:core][:qual] = mapq
       end
 
-      # returns a `Cigar` object.
+      # Get the Bam::Cigar object.
+      # @return [Bam::Cigar] cigar
       def cigar
         Cigar.new(LibHTS.bam_get_cigar(@bam1), @bam1[:core][:n_cigar])
       end
 
+      # Calculate query length from CIGAR.
+      # @return [Integer] query length
       def qlen
         LibHTS.bam_cigar2qlen(
           @bam1[:core][:n_cigar],
@@ -143,6 +163,8 @@ module HTS
         )
       end
 
+      # Calculate reference length from CIGAR.
+      # @return [Integer] reference length
       def rlen
         LibHTS.bam_cigar2rlen(
           @bam1[:core][:n_cigar],
@@ -150,7 +172,8 @@ module HTS
         )
       end
 
-      # return the read sequence
+      # Get the sequence. (a.k.a SEQ)
+      # @return [String] sequence
       def seq
         r = LibHTS.bam_get_seq(@bam1)
         seq = String.new
@@ -161,11 +184,15 @@ module HTS
       end
       alias sequence seq
 
+      # Get the length of the query sequence.
+      # @return [Integer] query length
       def len
         @bam1[:core][:l_qseq]
       end
 
-      # return only the base of the requested index "i" of the query sequence.
+      # Get the base of the requested index "i" of the query sequence.
+      # @param [Integer] i index
+      # @return [String] base
       def base(n)
         n += @bam1[:core][:l_qseq] if n < 0
         return "." if (n >= @bam1[:core][:l_qseq]) || (n < 0) # eg. base(-1000)
@@ -174,13 +201,23 @@ module HTS
         SEQ_NT16_STR[LibHTS.bam_seqi(r, n)]
       end
 
-      # return the base qualities
+      # Get the base qualities.
+      # @return [Array] base qualities
       def qual
         q_ptr = LibHTS.bam_get_qual(@bam1)
         q_ptr.read_array_of_uint8(@bam1[:core][:l_qseq])
       end
 
-      # return only the base quality of the requested index "i" of the query sequence.
+      # Get the base qualities as a string. (a.k.a QUAL)
+      # ASCII of base quality + 33.
+      # @return [String] base qualities
+      def qual_string
+        qual.map { |q| (q + 33).chr }.join
+      end
+
+      # Get the base quality of the requested index "i" of the query sequence.
+      # @param [Integer] i index
+      # @return [Integer] base quality
       def base_qual(n)
         n += @bam1[:core][:l_qseq] if n < 0
         return 0 if (n >= @bam1[:core][:l_qseq]) || (n < 0) # eg. base_qual(-1000)
@@ -189,7 +226,8 @@ module HTS
         q_ptr.get_uint8(n)
       end
 
-      # returns a `Flag` object.
+      # Get Bam::Flag object of the alignment.
+      # @return [Bam::Flag] flag
       def flag
         Flag.new(@bam1[:core][:flag])
       end
@@ -205,7 +243,9 @@ module HTS
         end
       end
 
-      # retruns the auxillary fields.
+      # Get the auxiliary data.
+      # @param [String] key tag name
+      # @return [String] value
       def aux(key = nil)
         aux = Aux.new(self)
         if key
@@ -230,6 +270,7 @@ module HTS
         end
       end
 
+      # @return [String] a string representation of the alignment.
       def to_s
         kstr = LibHTS::KString.new
         raise "Failed to format bam record" if LibHTS.sam_format1(@header.struct, @bam1, kstr) == -1