htslib 0.2.6 → 0.2.9

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 35551ff5a5cd81937063363c243bc2ac3d12ce09ec69697cd0eedd93945526bd
-  data.tar.gz: ec16e4f3cee66c50a582c4580d6cf266c0f349f8f6bc985864eda3d41b41e412
+  metadata.gz: c98f3937d65f091e9e834060f3a94578034d1bf55e7bc372e5ed388e618a6da4
+  data.tar.gz: 8e180044fef210935695bc60c0352b747b570b705f3ce9318b7538d19ddaf645
 SHA512:
-  metadata.gz: f534244d9b6dfd8a741cf21f639205207df612a3107ce563640db7dadd591c140dba92cf6eaa68ae1a440ab9b958697d0a04fac8217ff0ca7dcea796557fe30b
-  data.tar.gz: ac305efbbdd989f8f44e691499a0aa978aa4ed17e24546fd48826a25c2591a8f70a8ec0411456e123248597c518e4c41dc057fbd6738be49b96fc5198ba643cd
+  metadata.gz: 5324913bdbe97580fee1e54b6268f22aed2a0b84d449c8a4f62529981c82a57d6ae5cea7cc77ff1be7332425eea259bfe5daa1107b6144e36ef40bb4997dd28f
+  data.tar.gz: 7b230700951223aeda09d64fac7cca44d734d54ec03a7836c5a0fb036e552854db80b9ecaf57aa478e317c684e4712f2f03244e02474fd2ea53a4b93346d1371

data/README.md

@@ -1,7 +1,7 @@
 # ruby-htslib
 
 [![Gem Version](https://badge.fury.io/rb/htslib.svg)](https://badge.fury.io/rb/htslib)
-![CI](https://github.com/kojix2/ruby-htslib/workflows/CI/badge.svg)
+[![test](https://github.com/kojix2/ruby-htslib/actions/workflows/ci.yml/badge.svg)](https://github.com/kojix2/ruby-htslib/actions/workflows/ci.yml)
 [![The MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE.txt)
 [![DOI](https://zenodo.org/badge/247078205.svg)](https://zenodo.org/badge/latestdoi/247078205)
 [![Docs Stable](https://img.shields.io/badge/docs-stable-blue.svg)](https://rubydoc.info/gems/htslib)
@@ -17,7 +17,7 @@ Ruby-htslib is the [Ruby](https://www.ruby-lang.org) bindings to [HTSlib](https:
   - Ubuntu : `apt install libhts-dev`
   - macOS : `brew install htslib`
   - Windows : [mingw-w64-htslib](https://packages.msys2.org/base/mingw-w64-htslib) is automatically fetched when installing the gem ([RubyInstaller](https://rubyinstaller.org) only).
-  - Build from source code (see Development section)
+  - Build from source code (see the Development section)
 
 ## Installation
 
@@ -30,10 +30,10 @@ will automatically detect the location of the shared library. If pkg-config does
 Alternatively, you can specify the directory of the shared library by setting the environment variable `HTSLIBDIR`.
 
 ```sh
-export HTSLIBDIR="/your/path/to/htslib" # libhts.so
+export HTSLIBDIR="/your/path/to/htslib" # Directory where libhts.so is located
 ```
 
-ruby-htslib also works on Windows; if you use RubyInstaller, htslib will be prepared automatically.
+ruby-htslib also works on Windows. If you use RubyInstaller, htslib will be prepared automatically.
 
 ## Usage
 
@@ -64,12 +64,24 @@ end
 bam.close
 ```
 
+With a block
+
+```ruby
+HTS::Bam.open("test/fixtures/moo.bam") do |bam|
+  bam.each do |r|
+    puts r.to_s
+  end
+end
+```
+
 ### HTS::Bcf - VCF / BCF - Variant Call Format file
 
 Reading fields
 
 ```ruby
-bcf = HTS::Bcf.open("b.bcf")
+require 'htslib'
+
+bcf = HTS::Bcf.open("test/fixtures/test.bcf")
 
 bcf.each do |r|
   p chrom:  r.chrom,
@@ -86,27 +98,38 @@ end
 bcf.close
 ```
 
+With a block
+
+```ruby
+HTS::Bcf.open("test/fixtures/test.bcf") do |bcf|
+  bcf.each do |r|
+    puts r.to_s
+  end
+end
+```
+
 ### HTS::Faidx - FASTA / FASTQ - Nucleic acid sequence
 
 ```ruby
-fa = HTS::Faidx.open("c.fa")
-fa.seq("chr1:1-10")
+fa = HTS::Faidx.open("test/fixtures/moo.fa")
+fa.seq("chr1:1-10") # => CGCAACCCGA # 1-based
+fa.close
 ```
 
-### HTS::Tbx - Tabix - gff, bed, sam, vcf
+### HTS::Tabix - GFF / BED - TAB-delimited genome position file
 
 ```ruby
-tb = HTS::Tbx.open("test.vcf.gz")
-tb.query("chr1", 10000, 20000) do |line|
-  p line
+tb = HTS::Tabix.open("test/fixtures/test.vcf.gz")
+tb.query("poo", 2000, 3000) do |line|
+  puts line.join("\t")
 end
+tb.close
 ```
 
-Note: Faidx or Tbx should not be explicitly closed. See [#27](https://github.com/kojix2/ruby-htslib/issues/27)
-
 ### Low-level API
 
-`HTS::LibHTS` provides native C functions.
+Middle architectural layer between high-level Ruby code and low-level C code.
+`HTS::LibHTS` provides native C functions using [Ruby-FFI](https://github.com/ffi/ffi). 
 
 ```ruby
 require 'htslib'
@@ -117,26 +140,31 @@ p b[:category]
 p b[:format]
 ```
 
-Macro functions
+The low-level API makes it possible to perform detailed operations, such as calling CRAM-specific functions.
 
-htslib has a lot of macro functions for speed. Ruby-FFI cannot call C macro functions. However, essential functions are reimplemented in Ruby, and you can call them.
+#### Macro functions
 
-Structs
+HTSlib is designed to improve performance with many macro functions. However, it is not possible to call C macro functions directly from Ruby-FFI. To overcome this, important macro functions have been re-implemented in Ruby, allowing them to be called in the same way as native functions.
 
-Only a small number of C structs are implemented with FFI's ManagedStruct, which frees memory when Ruby's garbage collection fires. Other structs will need to be freed manually.
+#### Garbage Collection and Memory Freeing
+
+A small number of commonly used structs, such as `Bam1` and `Bcf1`, are implemented using FFI's `ManagedStruct`. This allows for automatic memory release when Ruby's garbage collection is triggered. On the other hand, other structs are implemented using `FFI::Struct`, and they will require manual memory release.
 
 ### Need more speed?
 
-Try Crystal. [HTS.cr](https://github.com/bio-cr/hts.cr) is implemented in Crystal language and provides an API compatible with ruby-htslib. Crystal language is not as flexible as Ruby language. You can not use `eval` methods and must always be careful with the types. Writing one-time scripts in Crystal may be less fun. However, if you have a clear idea of what you want to do in your mind, have already written code in Ruby, and need to run them over and over, try creating a command line tool in Crystal. The Crystal language is as fast as the Rust and C languages. It will give you incredible power to make tools.
+Try Crystal. [HTS.cr](https://github.com/bio-cr/hts.cr) is implemented in Crystal language and provides an API compatible with ruby-htslib. 
 
 ## Documentation
 
 - [TUTORIAL.md](TUTORIAL.md)
 - [API Documentation (develop branch)](https://kojix2.github.io/ruby-htslib/)
-- [RubyDoc.info - HTSlib](https://rdoc.info/gems/htslib)
+- [API Documentation (released gem)](https://rubydoc.info/gems/htslib)
 
 ## Development
 
+#### Compile from source code
+
+[GNU Autotools](https://en.wikipedia.org/wiki/GNU_Autotools) is required to compile htslib.
 To get started with development:
 
 ```sh
@@ -147,21 +175,41 @@ bundle exec rake htslib:build
 bundle exec rake test
 ```
 
-[GNU Autotools](https://en.wikipedia.org/wiki/GNU_Autotools) is required to compile htslib.
+#### Macro functions are reimplemented
 
 HTSlib has many macro functions. These macro functions cannot be called from FFI and must be reimplemented in Ruby.
 
-- Use the new version of Ruby to take full advantage of Ruby's potential.
-  - This is possible because we have a small number of users.
-- Remain compatible with [HTS.cr](https://github.com/bio-cr/hts.cr).
-  - The most challenging part is the return value. In the Crystal language, methods are expected to return only one type. On the other hand, in the Ruby language, methods that return multiple classes are very common. For example, in the Crystal language, the compiler gets confused if the return value is one of six types: Int32, Int64, Float32, Float64, Nil, or String. In fact Crystal allows you to do that. But the code gets a little messy. In Ruby, this is very common and doesn't cause any problems.
-  - Ruby and Crystal are languages that use garbage collection. However, the memory release policy for allocated C structures is slightly different: in Ruby-FFI, you can define a `self.release` method in `FFI::Struct`. This method is called when GC. So you don't have to worry about memory in high-level APIs like Bam::Record or Bcf::Record, etc. Crystal requires you to define a finalize method on each class. So you need to define it in Bam::Record or Bcf::Record.
+#### Use the latest Ruby
+
+Use Ruby 3 or newer to take advantage of new features. This is possible because we have a small number of users.
+
+#### Keep compatibility with Crystal language
+
+Compatibility with Crystal language is important for Ruby-htslib development. 
+
+- [HTS.cr](https://github.com/bio-cr/hts.cr) - HTSlib bindings for Crystal
+
+Return value
+
+The most challenging part is the return value. In the Crystal language, methods are expected to return only one type. On the other hand, in the Ruby language, methods that return multiple classes are very common. For example, in the Crystal language, the compiler gets confused if the return value is one of six types: Int32, Int64, Float32, Float64, Nil, or String. In fact Crystal allows you to do that. But the code gets a little messy. In Ruby, this is very common and doesn't cause any problems.
+
+Memory management
+
+Ruby and Crystal are languages that use garbage collection. However, the memory release policy for allocated C structures is slightly different: in Ruby-FFI, you can define a `self.release` method in `FFI::Struct`. This method is called when GC. So you don't have to worry about memory in high-level APIs like Bam::Record or Bcf::Record, etc. Crystal requires you to define a finalize method on each class. So you need to define it in Bam::Record or Bcf::Record.
+
+Macro functions
+
+In ruby-htslib, C macro functions are added to `LibHTS`, but in Crystal, `LibHTS` is a Lib, so methods cannot be added. methods are added to `LibHTS2`.
+
+#### Naming convention
+
+If you are not sure about the naming of a method, follow the Rust-htslib API. This is a very weak rule. if a more appropriate name is found later in Ruby, it will replace it.
 
-Method naming generally follows the Rust-htslib API.
+#### Support for bitfields of structures
 
-#### FFI Extensions
+Since Ruby-FFI does not support structure bit fields, the following extensions are used.
 
-- [ffi-bitfield](https://github.com/kojix2/ffi-bitfield) : Extension of Ruby-FFI to support bitfields.
+- [ffi-bitfield](https://github.com/kojix2/ffi-bitfield) - Extension of Ruby-FFI to support bitfields.
 
 #### Automatic validation
 
@@ -180,7 +228,7 @@ Ruby-htslib is a library under development, so even minor improvements like typo
 - [financial contributions](https://github.com/sponsors/kojix2)
 
 ```markdown
-# Ownership and Commitment Rights
+# Ownership and Commit Rights
 
 Do you need commit rights to the ruby-htslib repository?
 Do you want to get admin rights and take over the project?

data/TUTORIAL.md

@@ -38,7 +38,7 @@ class Bcf{
   +each() Enumerable
   +query()
 }
-class Tbx~Hts~{
+class Tabix~Hts~{
   +@hts_file : FFI::Struct
 }
 class `Bam::Header`{
@@ -46,32 +46,46 @@ class `Bam::Header`{
   +struct()
   +target_count()
   +target_names()
+  +name2tid()
+  +tid2name()
   +to_s()
 }
 class `Bam::Record` {
   +@bam1 : FFI::Struct
   +@header : Bam::Header
   +struct()
+  +qname()
+  +qname=()
   +tid()
   +tid=()
   +mtid()
   +mtid=()
   +pos()
   +pos=()
-  +mpos()
-  +mpos=()
+  +mpos() +mate_pos()
+  +mpos=() +mate_pos=()
   +bin()
   +bin=()
-  +qname()
-  +flag()
-  +chorm()
+  +endpos()
+  +chorm() +contig()
+  +mate_chrom() +mate_contig()
+  +strand()
+  +mate_strand()
+  +isize() +insert_size()
+  +isize=() +insert_size=()
   +mapq()
+  +mapq=()
   +cigar()
-  +mate_chrom()
-  +isize()
-  +seq()
+  +qlen()
+  +rlen()
+  +seq() +sequence()
+  +len()
+  +base(n)
   +qual()
   +qual_string()
+  +base_qual(n)
+  +flag()
+  +flag=()
   +aux()
   +to_s()
 }
@@ -91,12 +105,19 @@ class `Bcf::Record`{
   +@bcf1 : FFI::Struct
   +@header : Bcf::Header
   +struct()
+  +rid()
+  +rid=()
   +chrom()
   +pos()
+  +pos=()
   +id()
-  +qual()
+  +id=()
+  +clear_id()
   +ref()
   +alt()
+  +alleles()
+  +qual()
+  +qual=()
   +filter()
   +info()
   +format()
@@ -164,6 +185,20 @@ class Faidx{
 
 ```
 
+## Installation
+
+```
+gem install htslib
+```
+
+You can check which shared libraries are used by ruby-htslib as follows
+
+```ruby
+require "htslib"
+puts HTS.lib_path
+# => "/home/kojix2/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/htslib-0.2.6/vendor/libhts.so"
+```
+
 ## HTS::Bam - SAM / BAM / CRAM - Sequence Alignment Map file
 
 Reading fields
@@ -194,9 +229,8 @@ bam.close
 Open with block
 
 ```ruby
-HTS::Bam.open("test/fixtures/moo.bam") do |b|
-  b.
-  do |r|
+HTS::Bam.open("test/fixtures/moo.bam") do |bam|
+  bam.each do |record|
     # ...
   end
 end
@@ -220,6 +254,18 @@ in.close
 out.close
 ```
 
+Create index
+
+```ruby
+HTS::Bam.open("foo.bam", build_index: true)
+```
+
+```
+b = HTS::Bam.open("foo.bam")
+            .build_index
+            .load_index
+```
+
 ## HTS::Bcf - VCF / BCF - Variant Call Format file
 
 Reading fields
@@ -245,8 +291,8 @@ bcf.close
 Open with block
 
 ```ruby
-HTS::Bcf.open("b.bcf") do |b|
-  b.each do |r|
+HTS::Bcf.open("b.bcf") do |bcf|
+  bcf.each do |record|
     # ...
   end
 end

data/lib/hts/bam/auxi.rb

@@ -3,8 +3,11 @@
 # Q. Why is the file name auxi.rb and not aux.rb?
 #
 # A. This is for compatibility with Windows.
+#
 # In Windows, aux is a reserved word
-# You cannot create a file named aux. Eww!
+# You cannot create a file named aux.
+#
+# What?! That's crazy!
 
 module HTS
   class Bam < Hts
@@ -15,6 +18,7 @@ module HTS
     # Ruby's Aux class references a part of it. There is no one-to-one
     # correspondence between C structures and Ruby's Aux class.
     class Aux
+      include Enumerable
       attr_reader :record
 
       def initialize(record)
@@ -26,27 +30,10 @@ module HTS
       # which provides methods like `get_int`, `get_float`, etc.
       # I think they are better than `fetch_int`` and `fetch_float`.
       def get(key, type = nil)
-        aux = LibHTS.bam_aux_get(@record.struct, key)
-        return nil if aux.null?
-
-        type = type ? type.to_s : aux.read_string(1)
+        aux_ptr = LibHTS.bam_aux_get(@record.struct, key)
+        return nil if aux_ptr.null?
 
-        # A (character), B (general array),
-        # f (real number), H (hexadecimal array),
-        # i (integer), or Z (string).
-
-        case type
-        when "i", "I", "c", "C", "s", "S"
-          LibHTS.bam_aux2i(aux)
-        when "f", "d"
-          LibHTS.bam_aux2f(aux)
-        when "Z", "H"
-          LibHTS.bam_aux2Z(aux)
-        when "A" # char
-          LibHTS.bam_aux2A(aux).chr
-        else
-          raise NotImplementedError, "type: #{t}"
-        end
+        get_ruby_aux(aux_ptr, type)
       end
 
       # For compatibility with HTS.cr.
@@ -67,6 +54,80 @@ module HTS
       def [](key)
         get(key)
       end
+
+      def first
+        aux_ptr = first_pointer
+        return nil if aux_ptr.null?
+
+        get_ruby_aux(aux_ptr)
+      end
+
+      def each
+        return enum_for(__method__) unless block_given?
+
+        aux_ptr = first_pointer
+        return nil if aux_ptr.null?
+
+        loop do
+          yield get_ruby_aux(aux_ptr)
+          aux_ptr = LibHTS.bam_aux_next(@record.struct, aux_ptr)
+          break if aux_ptr.null?
+        end
+      end
+
+      def to_h
+        h = {}
+        aux_ptr = first_pointer
+        return h if aux_ptr.null?
+
+        loop do
+          key = FFI::Pointer.new(aux_ptr.address - 2).read_string(2)
+          h[key] = get_ruby_aux(aux_ptr)
+          aux_ptr = LibHTS.bam_aux_next(@record.struct, aux_ptr)
+          break if aux_ptr.null?
+        end
+        h
+      end
+
+      private
+
+      def first_pointer
+        LibHTS.bam_aux_first(@record.struct)
+      end
+
+      def get_ruby_aux(aux_ptr, type = nil)
+        type = type ? type.to_s : aux_ptr.read_string(1)
+
+        # A (character), B (general array),
+        # f (real number), H (hexadecimal array),
+        # i (integer), or Z (string).
+
+        case type
+        when "i", "I", "c", "C", "s", "S"
+          LibHTS.bam_aux2i(aux_ptr)
+        when "f", "d"
+          LibHTS.bam_aux2f(aux_ptr)
+        when "Z", "H"
+          LibHTS.bam_aux2Z(aux_ptr)
+        when "A" # char
+          LibHTS.bam_aux2A(aux_ptr).chr
+        when "B" # array
+          t2 = aux_ptr.read_string(2)[1] # just a little less efficient
+          l = LibHTS.bam_auxB_len(aux_ptr)
+          case t2
+          when "c", "C", "s", "S", "i", "I"
+            # FIXME : Not efficient.
+            Array.new(l) { |i| LibHTS.bam_auxB2i(aux_ptr, i) }
+          when "f", "d"
+            # FIXME : Not efficient.
+            Array.new(l) { |i| LibHTS.bam_auxB2f(aux_ptr, i) }
+          else
+            raise NotImplementedError, "type: #{type} #{t2}"
+          end
+        else
+          raise NotImplementedError, "type: #{type}"
+        end
+      end
     end
   end
 end

data/lib/hts/bam/flag.rb

@@ -69,27 +69,29 @@ module HTS
       end
 
       def &(other)
-        Flag.new(@value & other.to_i)
+        self.class.new(@value & other.to_i)
       end
 
       def |(other)
-        Flag.new(@value | other.to_i)
+        self.class.new(@value | other.to_i)
       end
 
       def ^(other)
-        Flag.new(@value ^ other.to_i)
+        self.class.new(@value ^ other.to_i)
       end
 
       def ~
-        Flag.new(~@value)
+        # FIXME: Only 12bits are used for flags
+        # The result is different from the Crystal version.
+        self.class.new(~@value)
       end
 
       def <<(f)
-        Flag.new(@value << f.to_i)
+        self.class.new(@value << f.to_i)
       end
 
       def >>(other)
-        Flag.new(@value >> other.to_i)
+        self.class.new(@value >> other.to_i)
       end
 
       def to_i

data/lib/hts/bam/header.rb

@@ -6,8 +6,8 @@ module HTS
   class Bam < Hts
     # A class for working with alignment header.
     class Header
-      def self.parse(str)
-        new(LibHTS.sam_hdr_parse(str.size, str))
+      def self.parse(text)
+        new(LibHTS.sam_hdr_parse(text.size, text))
       end
 
       def initialize(arg = nil)
@@ -21,6 +21,8 @@ module HTS
         else
           raise TypeError, "Invalid argument"
         end
+
+        yield self if block_given?
       end
 
       def struct
@@ -31,11 +33,23 @@ module HTS
         @sam_hdr.to_ptr
       end
 
+      def targets
+        Array.new(target_count) do |i|
+          name = LibHTS.sam_hdr_tid2name(@sam_hdr, i)
+          len = LibHTS.sam_hdr_tid2len(@sam_hdr, i)
+          { name:, len: }
+        end
+      end
+
       def target_count
         # FIXME: sam_hdr_nref
         @sam_hdr[:n_targets]
       end
 
+      def target_name(tid)
+        tid2name(tid)
+      end
+
       def target_names
         Array.new(target_count) do |i|
           LibHTS.sam_hdr_tid2name(@sam_hdr, i)
@@ -48,15 +62,20 @@ module HTS
         end
       end
 
-      # experimental
-      def add_lines(str)
-        LibHTS.sam_hdr_add_lines(@sam_hdr, str, 0)
+      def write(...)
+        add_lines(...)
       end
 
       # experimental
-      def add_line(type, *args)
-        args = args.flat_map { |arg| [:string, arg] }
-        LibHTS.sam_hdr_add_line(@sam_hdr, type, *args, :pointer, FFI::Pointer::NULL)
+      def <<(obj)
+        case obj
+        when Array, Hash
+          args = obj.flatten(-1).map { |i| i.to_a if i.is_a?(Hash) }
+          add_line(*args)
+        else
+          add_lines(obj.to_s)
+        end
+        self
       end
 
       # experimental
@@ -87,8 +106,31 @@ module HTS
         LibHTS.sam_hdr_str(@sam_hdr)
       end
 
+      # experimental
+      def get_tid(name)
+        name2tid(name)
+      end
+
       private
 
+      def name2tid(name)
+        LibHTS.sam_hdr_name2tid(@sam_hdr, name)
+      end
+
+      def tid2name(tid)
+        LibHTS.sam_hdr_tid2name(@sam_hdr, tid)
+      end
+
+      def add_lines(str)
+        LibHTS.sam_hdr_add_lines(@sam_hdr, str, 0)
+      end
+
+      def add_line(*args)
+        type = args.shift
+        args = args.flat_map { |arg| [:string, arg] }
+        LibHTS.sam_hdr_add_line(@sam_hdr, type, *args, :pointer, FFI::Pointer::NULL)
+      end
+
       def initialize_copy(orig)
         @sam_hdr = LibHTS.sam_hdr_dup(orig.struct)
       end