htslib 0.2.5 → 0.2.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2a49c4758453c5c39915d9d17ccc33ce104699fca39c4083d4b2801767b655f1
-  data.tar.gz: 46ee49f6b452471e26afedfce70489cd24eebf86b1a18240bab125d465921fb8
+  metadata.gz: 35551ff5a5cd81937063363c243bc2ac3d12ce09ec69697cd0eedd93945526bd
+  data.tar.gz: ec16e4f3cee66c50a582c4580d6cf266c0f349f8f6bc985864eda3d41b41e412
 SHA512:
-  metadata.gz: 918c0919750eb3d436cf19b1011ffed0a8d0760b29e0a3c2e675a9efa563a3f5c3b9a9ae38232e67192df4d066c446bbc0443f00d07cab9ac0c3401b38bd3fb7
-  data.tar.gz: 343fbaec89a2e60a54feadd29856b26b772baa65ef52d181cd5845ed53a8e549504ddcdcc739dff716f9069a5b0b3e6f22f5136c2a612cc6014eb79ee225ab2b
+  metadata.gz: f534244d9b6dfd8a741cf21f639205207df612a3107ce563640db7dadd591c140dba92cf6eaa68ae1a440ab9b958697d0a04fac8217ff0ca7dcea796557fe30b
+  data.tar.gz: ac305efbbdd989f8f44e691499a0aa978aa4ed17e24546fd48826a25c2591a8f70a8ec0411456e123248597c518e4c41dc057fbd6738be49b96fc5198ba643cd

data/README.md

@@ -8,16 +8,16 @@
 
 Ruby-htslib is the [Ruby](https://www.ruby-lang.org) bindings to [HTSlib](https://github.com/samtools/htslib), a C library for high-throughput sequencing data formats. It allows you to read and write file formats commonly used in genomics, such as [SAM, BAM, VCF, and BCF](http://samtools.github.io/hts-specs/), in the Ruby language.
 
-:apple: Feel free to fork it! 
+:apple: Feel free to fork it!
 
 ## Requirements
 
-* [Ruby](https://github.com/ruby/ruby) 3.1 or above.
-* [HTSlib](https://github.com/samtools/htslib)
-  * Ubuntu : `apt install libhts-dev`
-  * macOS : `brew install htslib`
-  * Windows : [mingw-w64-htslib](https://packages.msys2.org/base/mingw-w64-htslib) is automatically fetched when installing the gem ([RubyInstaller](https://rubyinstaller.org) only).
-  * Build from source code (see Development section)
+- [Ruby](https://github.com/ruby/ruby) 3.1 or above.
+- [HTSlib](https://github.com/samtools/htslib)
+  - Ubuntu : `apt install libhts-dev`
+  - macOS : `brew install htslib`
+  - Windows : [mingw-w64-htslib](https://packages.msys2.org/base/mingw-w64-htslib) is automatically fetched when installing the gem ([RubyInstaller](https://rubyinstaller.org) only).
+  - Build from source code (see Development section)
 
 ## Installation
 
@@ -25,7 +25,7 @@ Ruby-htslib is the [Ruby](https://www.ruby-lang.org) bindings to [HTSlib](https:
 gem install htslib
 ```
 
-If you have installed htslib with apt on Ubuntu or homebrew on Mac, [pkg-config](https://github.com/ruby-gnome/pkg-config) 
+If you have installed htslib with apt on Ubuntu or homebrew on Mac, [pkg-config](https://github.com/ruby-gnome/pkg-config)
 will automatically detect the location of the shared library. If pkg-config does not work well, set `PKG_CONFIG_PATH`.
 Alternatively, you can specify the directory of the shared library by setting the environment variable `HTSLIBDIR`.
 
@@ -35,11 +35,11 @@ export HTSLIBDIR="/your/path/to/htslib" # libhts.so
 
 ruby-htslib also works on Windows; if you use RubyInstaller, htslib will be prepared automatically.
 
-## Overview
+## Usage
 
-### High-level API
+### HTS::Bam - SAM / BAM / CRAM - Sequence Alignment Map file
 
-HTS::Bam - SAM / BAM / CRAM - Sequence Alignment Map file
+Reading fields
 
 ```ruby
 require 'htslib'
@@ -64,7 +64,9 @@ end
 bam.close
 ```
 
-HTS::Bcf - VCF / BCF - Variant Call Format file
+### HTS::Bcf - VCF / BCF - Variant Call Format file
+
+Reading fields
 
 ```ruby
 bcf = HTS::Bcf.open("b.bcf")
@@ -84,27 +86,23 @@ end
 bcf.close
 ```
 
-<details>
-<summary><b>Faidx</b></summary>
+### HTS::Faidx - FASTA / FASTQ - Nucleic acid sequence
 
 ```ruby
 fa = HTS::Faidx.open("c.fa")
-
-fa.fetch("chr1:1-10")
-
-fa.close
+fa.seq("chr1:1-10")
 ```
 
-</details>
-
-<details>
-<summary><b>Tbx</b></summary>
+### HTS::Tbx - Tabix - gff, bed, sam, vcf
 
 ```ruby
-
+tb = HTS::Tbx.open("test.vcf.gz")
+tb.query("chr1", 10000, 20000) do |line|
+  p line
+end
 ```
 
-</details>
+Note: Faidx or Tbx should not be explicitly closed. See [#27](https://github.com/kojix2/ruby-htslib/issues/27)
 
 ### Low-level API
 
@@ -133,8 +131,9 @@ Try Crystal. [HTS.cr](https://github.com/bio-cr/hts.cr) is implemented in Crysta
 
 ## Documentation
 
-* [API Documentation (develop branch)](https://kojix2.github.io/ruby-htslib/)
-* [RubyDoc.info - HTSlib](https://rdoc.info/gems/htslib)
+- [TUTORIAL.md](TUTORIAL.md)
+- [API Documentation (develop branch)](https://kojix2.github.io/ruby-htslib/)
+- [RubyDoc.info - HTSlib](https://rdoc.info/gems/htslib)
 
 ## Development
 
@@ -152,36 +151,35 @@ bundle exec rake test
 
 HTSlib has many macro functions. These macro functions cannot be called from FFI and must be reimplemented in Ruby.
 
-* Use the new version of Ruby to take full advantage of Ruby's potential.
-  * This is possible because we have a small number of users.
-* Remain compatible with [HTS.cr](https://github.com/bio-cr/hts.cr).
-  * The most challenging part is the return value. In the Crystal language, methods are expected to return only one type. On the other hand, in the Ruby language, methods that return multiple classes are very common. For example, in the Crystal language, the compiler gets confused if the return value is one of six types: Int32, Int64, Float32, Float64, Nil, or String. In fact Crystal allows you to do that. But the code gets a little messy. In Ruby, this is very common and doesn't cause any problems.
-  * Ruby and Crystal are languages that use garbage collection. However, the memory release policy for allocated C structures is slightly different: in Ruby-FFI, you can define a `self.release` method in `FFI::Struct`. This method is called when GC. So you don't have to worry about memory in high-level APIs like Bam::Record or Bcf::Record, etc. Crystal requires you to define a finalize method on each class. So you need to define it in Bam::Record or Bcf::Record.
+- Use the new version of Ruby to take full advantage of Ruby's potential.
+  - This is possible because we have a small number of users.
+- Remain compatible with [HTS.cr](https://github.com/bio-cr/hts.cr).
+  - The most challenging part is the return value. In the Crystal language, methods are expected to return only one type. On the other hand, in the Ruby language, methods that return multiple classes are very common. For example, in the Crystal language, the compiler gets confused if the return value is one of six types: Int32, Int64, Float32, Float64, Nil, or String. In fact Crystal allows you to do that. But the code gets a little messy. In Ruby, this is very common and doesn't cause any problems.
+  - Ruby and Crystal are languages that use garbage collection. However, the memory release policy for allocated C structures is slightly different: in Ruby-FFI, you can define a `self.release` method in `FFI::Struct`. This method is called when GC. So you don't have to worry about memory in high-level APIs like Bam::Record or Bcf::Record, etc. Crystal requires you to define a finalize method on each class. So you need to define it in Bam::Record or Bcf::Record.
 
-Method naming generally follows the Rust-htslib API. 
+Method naming generally follows the Rust-htslib API.
 
 #### FFI Extensions
 
-* [ffi-bitfield](https://github.com/kojix2/ffi-bitfield) : Extension of Ruby-FFI to support bitfields.
+- [ffi-bitfield](https://github.com/kojix2/ffi-bitfield) : Extension of Ruby-FFI to support bitfields.
 
 #### Automatic validation
 
 In the `script` directory, there are several tools to help implement ruby-htslib. Scripts using c2ffi can check the coverage of htslib functions in Ruby-htslib. They are useful when new versions of htslib are released.
 
-* [c2ffi](https://github.com/rpav/c2ffi) is a tool to create JSON format metadata from C header files. 
+- [c2ffi](https://github.com/rpav/c2ffi) is a tool to create JSON format metadata from C header files.
 
 ## Contributing
 
 Ruby-htslib is a library under development, so even minor improvements like typo fixes are welcome! Please feel free to send us your pull requests.
 
-* [Report bugs](https://github.com/kojix2/ruby-htslib/issues)
-* Fix bugs and [submit pull requests](https://github.com/kojix2/ruby-htslib/pulls)
-* Write, clarify, or fix documentation
-* Suggest or add new features
-* [financial contributions](https://github.com/sponsors/kojix2)
+- [Report bugs](https://github.com/kojix2/ruby-htslib/issues)
+- Fix bugs and [submit pull requests](https://github.com/kojix2/ruby-htslib/pulls)
+- Write, clarify, or fix documentation
+- Suggest or add new features
+- [financial contributions](https://github.com/sponsors/kojix2)
 
-
-```
+```markdown
 # Ownership and Commitment Rights
 
 Do you need commit rights to the ruby-htslib repository?
@@ -195,8 +193,8 @@ One of the greatest joys of using a minor language like Ruby in bioinformatics i
 
 ## Links
 
-* [samtools/hts-spec](https://github.com/samtools/hts-specs)
-* [bioruby](https://github.com/bioruby/bioruby)
+- [samtools/hts-spec](https://github.com/samtools/hts-specs)
+- [bioruby](https://github.com/bioruby/bioruby)
 
 ## Funding support
 

data/TUTORIAL.md

@@ -0,0 +1,270 @@
+# Tutorial
+
+```mermaid
+%%{init:{'theme':'base'}}%%
+classDiagram
+Bam~Hts~ o-- `Bam::Header`
+Bam o-- `Bam::Record`
+`Bam::Record` o-- `Bam::Header`
+Bcf~Hts~ o-- `Bcf::Header`
+Bcf o-- `Bcf::Record`
+`Bcf::Record` o--`Bcf::Header`
+`Bam::Header` o-- `Bam::HeaderRecord`
+`Bcf::Header` o-- `Bcf::HeaderRecord`
+`Bam::Record` o-- Flag
+`Bam::Record` o-- Cigar
+`Bam::Record` o-- Aux
+`Bcf::Record` o-- Info
+`Bcf::Record` o-- Format
+class Bam{
+  +@hts_file : FFI::Struct
+  +@header : Bam::Header
+  +@file_name
+  +@index_name
+  +@mode
+  +struct()
+  +build_index()
+  +each() Enumerable
+  +query()
+}
+class Bcf{
+  +@hts_file : FFI::Struct
+  +@header : Bcf::Header
+  +@file_name
+  +@index_name
+  +@mode
+  +struct()
+  +build_index()
+  +each() Enumerable
+  +query()
+}
+class Tbx~Hts~{
+  +@hts_file : FFI::Struct
+}
+class `Bam::Header`{
+  +@sam_hdr : FFI::Struct
+  +struct()
+  +target_count()
+  +target_names()
+  +to_s()
+}
+class `Bam::Record` {
+  +@bam1 : FFI::Struct
+  +@header : Bam::Header
+  +struct()
+  +tid()
+  +tid=()
+  +mtid()
+  +mtid=()
+  +pos()
+  +pos=()
+  +mpos()
+  +mpos=()
+  +bin()
+  +bin=()
+  +qname()
+  +flag()
+  +chorm()
+  +mapq()
+  +cigar()
+  +mate_chrom()
+  +isize()
+  +seq()
+  +qual()
+  +qual_string()
+  +aux()
+  +to_s()
+}
+class `Aux` {
+  -@record : Bam::Record
+  +[]()
+  +get_int()
+  +get_float()
+  +get_string()
+}
+class `Bcf::Header`{
+  +@bcf_hdr : FFI::Struct
+  +struct()
+  +to_s()
+}
+class `Bcf::Record`{
+  +@bcf1 : FFI::Struct
+  +@header : Bcf::Header
+  +struct()
+  +chrom()
+  +pos()
+  +id()
+  +qual()
+  +ref()
+  +alt()
+  +filter()
+  +info()
+  +format()
+  +to_s()
+}
+class Flag {
+  +@value : Integer
+  +paired?()
+  +proper_pair?()
+  +unmapped?()
+  +mate_unmapped?()
+  +reverse?()
+  +mate_reverse?()
+  +read1?()
+  +read2?()
+  +secondary?()
+  +qcfail?()
+  +duplicate?()
+  +supplementary?()
+  +&()
+  +|()
+  +^()
+  +~()
+  +<<()
+  +>>()
+  +to_i()
+  +to_s()
+}
+class Info {
+  -@record : Bcf::Record
+  +[]()
+  +get_int()
+  +get_float()
+  +get_string()
+  +get_flag()
+  +fields()
+  +length() +size()
+  +to_h()
+  -info_ptr()
+}
+class Format {
+  -@record : Bcf::Record
+  +[]()\
+  +get_int()
+  +get_float()
+  +get_string()
+  +get_flag()
+  +fields()
+  +length() +size()
+  +to_h()
+  -format_ptr()
+}
+class Cigar {
+  -@array : Array
+  +each() Enumerable
+  +qlen()
+  +rlen()
+  +to_s()
+  +==()
+  +eql?()
+}
+class Faidx{
+  +@fai
+}
+
+```
+
+## HTS::Bam - SAM / BAM / CRAM - Sequence Alignment Map file
+
+Reading fields
+
+```ruby
+require 'htslib'
+
+bam = HTS::Bam.open("test/fixtures/moo.bam")
+
+bam.each do |r|
+  pp name: r.qname,
+     flag: r.flag,
+     chrm: r.chrom,
+     strt: r.pos + 1,
+     mapq: r.mapq,
+     cigr: r.cigar.to_s,
+     mchr: r.mate_chrom,
+     mpos: r.mpos + 1,
+     isiz: r.isize,
+     seqs: r.seq,
+     qual: r.qual_string,
+     MC:   r.aux("MC")
+end
+
+bam.close
+```
+
+Open with block
+
+```ruby
+HTS::Bam.open("test/fixtures/moo.bam") do |b|
+  b.
+  do |r|
+    # ...
+  end
+end
+```
+
+Writing
+
+```ruby
+in = HTS::Bam.open("foo.bam")
+out = HTS::Bam.open("bar.bam", "wb")
+
+out.header = in.header
+# out.write_header(in.header)
+
+in.each do |r|
+  out << r
+  # out.write(r)
+end
+
+in.close
+out.close
+```
+
+## HTS::Bcf - VCF / BCF - Variant Call Format file
+
+Reading fields
+
+```ruby
+bcf = HTS::Bcf.open("b.bcf")
+
+bcf.each do |r|
+  p chrom:  r.chrom,
+    pos:    r.pos,
+    id:     r.id,
+    qual:   r.qual.round(2),
+    ref:    r.ref,
+    alt:    r.alt,
+    filter: r.filter,
+    info:   r.info.to_h,
+    format: r.format.to_h
+end
+
+bcf.close
+```
+
+Open with block
+
+```ruby
+HTS::Bcf.open("b.bcf") do |b|
+  b.each do |r|
+    # ...
+  end
+end
+```
+
+Writing
+
+```ruby
+in = HTS::Bcf.open("foo.bcf")
+out = HTS::Bcf.open("bar.bcf", "wb")
+
+out.header = in.header
+# out.write_header(in.header)
+in.each do |r|
+  out << r
+  # out.write(r)
+end
+
+in.close
+out.close
+```

data/lib/hts/bam/auxi.rb

@@ -4,21 +4,32 @@
 #
 # A. This is for compatibility with Windows.
 # In Windows, aux is a reserved word
-# You cannot create a file named aux.
+# You cannot create a file named aux. Eww!
 
 module HTS
   class Bam < Hts
     # Auxiliary record data
+    #
+    # @noge Aux is a View object.
+    # The result of the alignment is assigned to the bam1 structure.
+    # Ruby's Aux class references a part of it. There is no one-to-one
+    # correspondence between C structures and Ruby's Aux class.
     class Aux
+      attr_reader :record
+
       def initialize(record)
         @record = record
       end
 
+      # @note Why is this method named "get" instead of "fetch"?
+      # This is for compatibility with the Crystal language
+      # which provides methods like `get_int`, `get_float`, etc.
+      # I think they are better than `fetch_int`` and `fetch_float`.
       def get(key, type = nil)
         aux = LibHTS.bam_aux_get(@record.struct, key)
         return nil if aux.null?
 
-        type ||= aux.read_string(1)
+        type = type ? type.to_s : aux.read_string(1)
 
         # A (character), B (general array),
         # f (real number), H (hexadecimal array),
@@ -38,6 +49,21 @@ module HTS
         end
       end
 
+      # For compatibility with HTS.cr.
+      def get_int(key)
+        get(key, "i")
+      end
+
+      # For compatibility with HTS.cr.
+      def get_float(key)
+        get(key, "f")
+      end
+
+      # For compatibility with HTS.cr.
+      def get_string(key)
+        get(key, "Z")
+      end
+
       def [](key)
         get(key)
       end

data/lib/hts/bam/cigar.rb

@@ -6,11 +6,31 @@ module HTS
     class Cigar
       include Enumerable
 
-      def initialize(pointer, n_cigar)
-        @n_cigar = n_cigar
-        # Read the pointer before the memory is changed.
-        # Especially when called from a block of `each` iterator.
-        @c = pointer.read_array_of_uint32(n_cigar)
+      # a uint32_t array (with 32 bits for every CIGAR op: length<<4|operation)
+      attr_accessor :array
+
+      # Create a new Cigar object from a string.
+      # @param [String] cigar_str
+      # The CIGAR string is converted to a uint32_t array in htslib.
+      def self.parse(str)
+        c = FFI::MemoryPointer.new(:pointer)
+        m = FFI::MemoryPointer.new(:size_t)
+        LibHTS.sam_parse_cigar(str, FFI::Pointer::NULL, c, m)
+        cigar_array = c.read_pointer.read_array_of_uint32(m.read(:size_t))
+        obj = new
+        obj.array = cigar_array
+        obj
+      end
+
+      def initialize(record = nil)
+        if record
+          # The record is used at initialization and is not retained after that.
+          bam1 = record.struct
+          n_cigar = bam1[:core][:n_cigar]
+          @array = LibHTS.bam_get_cigar(bam1).read_array_of_uint32(n_cigar)
+        else
+          @array = []
+        end
       end
 
       def to_s
@@ -20,12 +40,32 @@ module HTS
       def each
         return to_enum(__method__) unless block_given?
 
-        @c.each do |c|
+        @array.each do |c|
           op =  LibHTS.bam_cigar_opchr(c)
           len = LibHTS.bam_cigar_oplen(c)
           yield [op, len]
         end
       end
+
+      def qlen
+        a = FFI::MemoryPointer.new(:uint32, @array.size)
+        a.write_array_of_uint32(@array)
+        LibHTS.bam_cigar2qlen(@array.size, a)
+      end
+
+      def rlen
+        a = FFI::MemoryPointer.new(:uint32, @array.size)
+        a.write_array_of_uint32(@array)
+        LibHTS.bam_cigar2rlen(@array.size, a)
+      end
+
+      def ==(other)
+        other.is_a?(Cigar) && (@array == other.array)
+      end
+
+      def eql?(other)
+        other.is_a?(Cigar) && @array.eql?(other.array)
+      end
     end
   end
 end

data/lib/hts/bam/flag.rb

@@ -28,8 +28,6 @@ module HTS
       # BAM_FDUP           = 1024
       # BAM_FSUPPLEMENTARY = 2048
 
-      # TODO: Enabling bitwise operations?
-
       TABLE = { paired?: LibHTS::BAM_FPAIRED,
                 proper_pair?: LibHTS::BAM_FPROPER_PAIR,
                 unmapped?: LibHTS::BAM_FUNMAP,
@@ -43,16 +41,57 @@ module HTS
                 duplicate?: LibHTS::BAM_FDUP,
                 supplementary?: LibHTS::BAM_FSUPPLEMENTARY }.freeze
 
-      TABLE.each do |name, v|
+      # @!macro [attach] generate_flag_methods
+      #   @!method $1
+      #   @return [Boolean]
+      def self.generate(name)
         define_method(name) do
-          has_flag?(v)
+          (@value & TABLE[name]) != 0
         end
       end
+      private_class_method :generate
+
+      generate :paired?
+      generate :proper_pair?
+      generate :unmapped?
+      generate :mate_unmapped?
+      generate :reverse?
+      generate :mate_reverse?
+      generate :read1?
+      generate :read2?
+      generate :secondary?
+      generate :qcfail?
+      generate :duplicate?
+      generate :supplementary?
 
       def has_flag?(f)
         (@value & f) != 0
       end
 
+      def &(other)
+        Flag.new(@value & other.to_i)
+      end
+
+      def |(other)
+        Flag.new(@value | other.to_i)
+      end
+
+      def ^(other)
+        Flag.new(@value ^ other.to_i)
+      end
+
+      def ~
+        Flag.new(~@value)
+      end
+
+      def <<(f)
+        Flag.new(@value << f.to_i)
+      end
+
+      def >>(other)
+        Flag.new(@value >> other.to_i)
+      end
+
       def to_i
         @value
       end

data/lib/hts/bam/record.rb

@@ -151,12 +151,13 @@ module HTS
       # Get the Bam::Cigar object.
       # @return [Bam::Cigar] cigar
       def cigar
-        Cigar.new(LibHTS.bam_get_cigar(@bam1), @bam1[:core][:n_cigar])
+        Cigar.new(self)
       end
 
       # Calculate query length from CIGAR.
       # @return [Integer] query length
       def qlen
+        # cigar.qlen will be slower because it converts to a Ruby array.
         LibHTS.bam_cigar2qlen(
           @bam1[:core][:n_cigar],
           LibHTS.bam_get_cigar(@bam1)