htslib 0.0.3 → 0.0.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 30f42b474bc317136d665b00781fbfcb11caaf588b091e76bc86bf9cdf8d5e3f
-  data.tar.gz: d48e5f74fb0efed4de5af2955b0093d1eb7ef5ee7767bc7ee50bf3e296e7ce28
+  metadata.gz: c8048806df4c335ea698c8d3ad9e51e12644f0d32667d3db5fac59a004db75bf
+  data.tar.gz: 3d18183921f70b7b42dc5657195607c519c7d83c1e0e94b9b7a8f647d96f5504
 SHA512:
-  metadata.gz: 6c1bf27a8fdc04a4a9ba678923df5bb579439c286802a5d1f2a4e6f11d7102217eafa0e4e42c2fa853e9ee82c706756315a0a1d6f97c5b5fab58ee909add4eb0
-  data.tar.gz: 59219371057e45cf31951eda2dae250acaedd12c593c09fb08f19720818be514797c57e9b45be8e1f6ea60f2fbc79fe6038a1aced6ba66d8f39a544eb6516a0a
+  metadata.gz: f50e96a23cb75130c5aa463329cce3c2d5784ed75c608d4685d2662ee22fa93e55208c356bce0fee319028c6ae8e33b5b20d270a17fb63ca6e39a736f668a2e4
+  data.tar.gz: c223f9b61979ace926fa53b408e85fa6f2e127cdde6ffa5bcfa1ac0ad7ce6c9c1ac5570522021c61e82d27498473299b8875f7d633b24367ae99dd2494758556

data/README.md

@@ -6,17 +6,21 @@
 [![DOI](https://zenodo.org/badge/247078205.svg)](https://zenodo.org/badge/latestdoi/247078205)
 [![Docs Stable](https://img.shields.io/badge/docs-stable-blue.svg)](https://rubydoc.info/gems/htslib)
 
-:dna: [HTSlib](https://github.com/samtools/htslib) - high-throughput sequencing data manipulation - for Ruby
+:dna: [HTSlib](https://github.com/samtools/htslib) - for Ruby
+
+Ruby-htslib is the Ruby bindings to HTSlib, a C library for processing high throughput sequencing (HTS) data. 
+It will provide APIs to read and write file formats such as SAM, BAM, VCF, and BCF.
 
 :apple: Feel free to fork it out if you can develop it! 
 
 :bowtie: alpha stage.
-
 ## Requirements
 
-* [htslib](https://github.com/samtools/htslib)
+* [Ruby](https://github.com/ruby/ruby) 2.7 or above.
+* [HTSlib](https://github.com/samtools/htslib)
   * Ubuntu : `apt install libhts-dev`
   * macOS : `brew install htslib`
+  * Build from source code (see Development section)
 
 ## Installation
 
@@ -24,19 +28,19 @@
 gem install htslib
 ```
 
-If you installed htslib with Ubuntu/apt or Mac/homebrew, [pkg-config](https://github.com/ruby-gnome/pkg-config) will automatically detect the location of the shared library.
-
-Or you can set the environment variable `HTSLIBDIR`.
+If you have installed htslib with apt on Ubuntu or homebrew on Mac, [pkg-config](https://github.com/ruby-gnome/pkg-config) 
+will automatically detect the location of the shared library. 
+Alternatively, you can specify the directory of the shared library by setting the environment variable `HTSLIBDIR`.
 
 ```sh
 export HTSLIBDIR="/your/path/to/htslib" # libhts.so
 ```
 
-## Usage
+## Overview
 
 ### Low level API
 
-HTS::LibHTS
+`HTS::LibHTS` provides native functions. 
 
 ```ruby
 require 'htslib'
@@ -49,9 +53,11 @@ p b[:format]
 
 Note: Managed struct is not used in ruby-htslib. You may need to free the memory by yourself.
 
-### High level API
+### High level API (Plan)
 
-A high-level API based on [hts-python](https://github.com/quinlan-lab/hts-python) or [hts-nim](https://github.com/brentp/hts-nim) is under development. We will change and improve the API to make it better.
+`Cram` `Bam` `Bcf` `Faidx` `Tabix`
+
+A high-level API is under development. We will change and improve the API to make it better.
 
 ```ruby
 require 'htslib'
@@ -86,8 +92,16 @@ bundle exec rake htslib:build
 bundle exec rake test
 ```
 
-[c2ffi](https://github.com/rpav/c2ffi) : 
-I am trying to find a way to automatically generate a low-level API using c2ffi.
+We plan to actively use the new features of Ruby. Since the number of users is small, backward compatibility is not important.
+On the other hand, we will consider compatibility with [Crystal](https://github.com/bio-crystal/htslib.cr) to some extent.
+
+#### FFI Extensions
+
+* [ffi-bitfield](https://github.com/kojix2/ffi-bitfield) : Extension of Ruby-FFI to support bitfields.
+
+#### Automatic generation or automatic validation (Future plan)
+
++ [c2ffi](https://github.com/rpav/c2ffi) is a tool to create JSON format metadata from C header files. It is planned to use c2ffi to automatically generate bindings or tests.
 
 ## Contributing
 
@@ -97,12 +111,16 @@ Ruby-htslib is a library under development, so even small improvements like typo
 * Fix bugs and [submit pull requests](https://github.com/kojix2/ruby-htslib/pulls)
 * Write, clarify, or fix documentation
 * Suggest or add new features
+* [financial contributions](https://github.com/sponsors/kojix2)
 
 ## Links
 
 * [samtools/hts-spec](https://github.com/samtools/hts-specs)
-* [c2ffi](https://github.com/rpav/c2ffi)
+* [bioruby](https://github.com/bioruby/bioruby)
+
+## Funding support
 
+This work was supported partially by [Ruby Association Grant 2020](https://www.ruby.or.jp/en/news/20201022).
 ## License
 
 [MIT License](https://opensource.org/licenses/MIT).

data/lib/hts/bam/cigar.rb

@@ -7,20 +7,23 @@ module HTS
   class Bam
     class Cigar
       include Enumerable
-      OPS = "MIDNSHP=XB"
 
-      def initialize(cigar, n_cigar)
-        @c = cigar
+      def initialize(pointer, n_cigar)
+        @pointer = pointer
         @n_cigar = n_cigar
       end
 
+      def to_ptr
+        @pointer
+      end
+
       def to_s
         to_a.flatten.join
       end
 
       def each
         @n_cigar.times do |i|
-          c = @c[i].read_uint32
+          c = @pointer[i].read_uint32
           yield [LibHTS.bam_cigar_oplen(c),
                  LibHTS.bam_cigar_opchr(c)]
         end

data/lib/hts/bam/flag.rb

@@ -7,7 +7,9 @@ module HTS
   class Bam
     class Flag
       def initialize(flag_value)
-        @value = flag_value # tytpe check?
+        raise TypeError unless flag_value.is_a? Integer
+
+        @value = flag_value
       end
 
       attr_accessor :value
@@ -84,7 +86,7 @@ module HTS
       end
 
       def has_flag?(o)
-        (@value & o) != 0
+        @value[o] != 0
       end
     end
   end

data/lib/hts/bam/header.rb

@@ -6,21 +6,27 @@
 module HTS
   class Bam
     class Header
-      attr_reader :h
+      def initialize(pointer)
+        @sam_hdr = pointer
+      end
+
+      def struct
+        @sam_hdr
+      end
 
-      def initialize(h)
-        @h = h
+      def to_ptr
+        @sam_hdr.to_ptr
       end
 
       # FIXME: better name?
       def seqs
-        Array.new(@h[:n_targets]) do |i|
-          LibHTS.sam_hdr_tid2name(@h, i)
+        Array.new(@sam_hdr[:n_targets]) do |i|
+          LibHTS.sam_hdr_tid2name(@sam_hdr, i)
         end
       end
 
       def text
-        LibHTS.sam_hdr_str(@h)
+        LibHTS.sam_hdr_str(@sam_hdr)
       end
     end
   end

data/lib/hts/bam/record.rb

@@ -8,9 +8,17 @@ module HTS
     class Record
       SEQ_NT16_STR = "=ACMGRSVTWYHKDBN"
 
-      def initialize(bam1_t, bam_hdr_t)
-        @b = bam1_t
-        @h = bam_hdr_t
+      def initialize(bam1_t, header)
+        @bam1 = bam1_t
+        @header = header
+      end
+
+      def struct
+        @bam1
+      end
+
+      def to_ptr
+        @bam1.to_ptr
       end
 
       # def initialize_copy
@@ -23,7 +31,7 @@ module HTS
 
       # returns the query name.
       def qname
-        LibHTS.bam_get_qname(@b).read_string
+        LibHTS.bam_get_qname(@bam1).read_string
       end
 
       # Set (query) name.
@@ -33,48 +41,48 @@ module HTS
 
       # returns the tid of the record or -1 if not mapped.
       def tid
-        @b[:core][:tid]
+        @bam1[:core][:tid]
       end
 
       # returns the tid of the mate or -1 if not mapped.
       def mate_tid
-        @b[:core][:mtid]
+        @bam1[:core][:mtid]
       end
 
       # returns 0-based start position.
       def start
-        @b[:core][:pos]
+        @bam1[:core][:pos]
       end
 
       # returns end position of the read.
       def stop
-        LibHTS.bam_endpos @b
+        LibHTS.bam_endpos @bam1
       end
 
       # returns 0-based mate position
       def mate_start
-        @b[:core][:mpos]
+        @bam1[:core][:mpos]
       end
       alias mate_pos mate_start
 
       # returns the chromosome or '' if not mapped.
       def chrom
-        tid = @b[:core][:tid]
+        tid = @bam1[:core][:tid]
         return "" if tid == -1
 
-        LibHTS.sam_hdr_tid2name(@h, tid)
+        LibHTS.sam_hdr_tid2name(@header, tid)
       end
 
       # returns the chromosome of the mate or '' if not mapped.
       def mate_chrom
-        tid = @b[:core][:mtid]
+        tid = @bam1[:core][:mtid]
         return "" if tid == -1
 
-        LibHTS.sam_hdr_tid2name(@h, tid)
+        LibHTS.sam_hdr_tid2name(@header, tid)
       end
 
       def strand
-        LibHTS.bam_is_rev(@b) ? "-" : "+"
+        LibHTS.bam_is_rev(@bam1) ? "-" : "+"
       end
 
       # def start=(v)
@@ -83,38 +91,38 @@ module HTS
 
       # insert size
       def isize
-        @b[:core][:isize]
+        @bam1[:core][:isize]
       end
 
       # mapping quality
       def mapping_quality
-        @b[:core][:qual]
+        @bam1[:core][:qual]
       end
 
       # returns a `Cigar` object.
       def cigar
-        Cigar.new(LibHTS.bam_get_cigar(@b), @b[:core][:n_cigar])
+        Cigar.new(LibHTS.bam_get_cigar(@bam1), @bam1[:core][:n_cigar])
       end
 
       def qlen
         LibHTS.bam_cigar2qlen(
-          @b[:core][:n_cigar],
-          LibHTS.bam_get_cigar(@b)
+          @bam1[:core][:n_cigar],
+          LibHTS.bam_get_cigar(@bam1)
         )
       end
 
       def rlen
         LibHTS.bam_cigar2rlen(
-          @b[:core][:n_cigar],
-          LibHTS.bam_get_cigar(@b)
+          @bam1[:core][:n_cigar],
+          LibHTS.bam_get_cigar(@bam1)
         )
       end
 
       # return the read sequence
       def sequence
-        r = LibHTS.bam_get_seq(@b)
+        r = LibHTS.bam_get_seq(@bam1)
         seq = String.new
-        (@b[:core][:l_qseq]).times do |i|
+        (@bam1[:core][:l_qseq]).times do |i|
           seq << SEQ_NT16_STR[LibHTS.bam_seqi(r, i)]
         end
         seq
@@ -122,35 +130,42 @@ module HTS
 
       # return only the base of the requested index "i" of the query sequence.
       def base_at(n)
-        n += @b[:core][:l_qseq] if n < 0
-        return "." if (n >= @b[:core][:l_qseq]) || (n < 0) # eg. base_at(-1000)
+        n += @bam1[:core][:l_qseq] if n < 0
+        return "." if (n >= @bam1[:core][:l_qseq]) || (n < 0) # eg. base_at(-1000)
 
-        r = LibHTS.bam_get_seq(@b)
+        r = LibHTS.bam_get_seq(@bam1)
         SEQ_NT16_STR[LibHTS.bam_seqi(r, n)]
       end
 
       # return the base qualities
       def base_qualities
-        q_ptr = LibHTS.bam_get_qual(@b)
-        q_ptr.read_array_of_uint8(@b[:core][:l_qseq])
+        q_ptr = LibHTS.bam_get_qual(@bam1)
+        q_ptr.read_array_of_uint8(@bam1[:core][:l_qseq])
       end
 
       # return only the base quality of the requested index "i" of the query sequence.
       def base_quality_at(n)
-        n += @b[:core][:l_qseq] if n < 0
-        return 0 if (n >= @b[:core][:l_qseq]) || (n < 0) # eg. base_quality_at(-1000)
+        n += @bam1[:core][:l_qseq] if n < 0
+        return 0 if (n >= @bam1[:core][:l_qseq]) || (n < 0) # eg. base_quality_at(-1000)
 
-        q_ptr = LibHTS.bam_get_qual(@b)
+        q_ptr = LibHTS.bam_get_qual(@bam1)
         q_ptr.get_uint8(n)
       end
 
       def flag_str
-        LibHTS.bam_flag2str(@b[:core][:flag])
+        LibHTS.bam_flag2str(@bam1[:core][:flag])
       end
 
       # returns a `Flag` object.
       def flag
-        Flag.new(@b[:core][:flag])
+        Flag.new(@bam1[:core][:flag])
+      end
+
+      def to_s
+        kstr = LibHTS::KString.new
+        raise "Failed to format bam record" if LibHTS.sam_format1(@header.struct, @bam1, kstr) == -1
+
+        kstr[:s]
       end
 
       # TODO:

data/lib/hts/bam.rb

@@ -7,75 +7,104 @@ require_relative "bam/header"
 require_relative "bam/cigar"
 require_relative "bam/flag"
 require_relative "bam/record"
+require_relative "utils/open_method"
 
 module HTS
   class Bam
     include Enumerable
-    attr_reader :file_path, :mode, :htf, :header
+    extend Utils::OpenMethod
+
+    attr_reader :file_path, :mode, :header
+    # HtfFile is FFI::BitStruct
+    attr_reader :htf_file
+
+    class << self
+      alias open new
+    end
 
     def initialize(file_path, mode = "r", create_index: nil)
       file_path = File.expand_path(file_path)
 
-      raise("No such SAM/BAM file - #{file_path}") unless File.exist?(file_path)
+      unless File.exist?(file_path)
+        message = "No such SAM/BAM file - #{file_path}"
+        raise message
+      end
 
       @file_path = file_path
       @mode      = mode
-      @htf       = LibHTS.hts_open(@file_path, mode)
-      @header    = Bam::Header.new(LibHTS.sam_hdr_read(htf))
+      @htf_file  = LibHTS.hts_open(file_path, mode)
+      @header    = Bam::Header.new(LibHTS.sam_hdr_read(htf_file))
+
       # FIXME: should be defined here?
-      @b         = LibHTS.bam_init1
+      @bam1      = LibHTS.bam_init1
 
       # read
       if mode[0] == "r"
         # load index
-        @idx = LibHTS.sam_index_load(htf, file_path)
+        @idx = LibHTS.sam_index_load(htf_file, file_path)
         # create index
         if create_index || (@idx.null? && create_index.nil?)
           warn "Create index for #{file_path}"
           LibHTS.sam_index_build(file_path, -1)
-          @idx = LibHTS.sam_index_load(@htf, @file_path)
+          @idx = LibHTS.sam_index_load(htf_file, file_path)
         end
       else
         # FIXME: implement
         raise "not implemented yet."
       end
+
+      # IO like API
+      if block_given?
+        begin
+          yield self
+        ensure
+          close
+        end
+      end
+    end
+
+    def struct
+      htf_file
+    end
+
+    def to_ptr
+      htf_file.to_ptr
     end
 
     def write(alns)
       alns.each do
-        LibHTS.sam_write1(htf, header, alns.b) > 0 || raise
+        LibHTS.sam_write1(htf_file, header, alns.b) > 0 || raise
       end
     end
 
     # Close the current file.
     def close
-      LibHTS.hts_close(htf)
+      LibHTS.hts_close(htf_file)
     end
 
     # Flush the current file.
     def flush
-      raise
-      # LibHTS.bgzf_flush(@htf.fp.bgzf)
+      # LibHTS.bgzf_flush(@htf_file.fp.bgzf)
     end
 
     def each(&block)
       # Each does not always start at the beginning of the file.
       # This is the common behavior of IO objects in Ruby.
       # This may change in the future.
-      while LibHTS.sam_read1(htf, header.h, @b) > 0
-        record = Record.new(@b, header.h)
+      while LibHTS.sam_read1(htf_file, header, @bam1) > 0
+        record = Record.new(@bam1, header)
         block.call(record)
       end
     end
 
     # query [WIP]
     def query(region)
-      qiter = LibHTS.sam_itr_querys(@idx, header.h, region)
+      qiter = LibHTS.sam_itr_querys(@idx, header, region)
       begin
-        slen = LibHTS.sam_itr_next(htf, qiter, @b)
+        slen = LibHTS.sam_itr_next(htf_file, qiter, @bam1)
         while slen > 0
-          yield Record.new(@b, header.h)
-          slen = LibHTS.sam_itr_next(htf, qiter, @b)
+          yield Record.new(@bam1, header)
+          slen = LibHTS.sam_itr_next(htf_file, qiter, @bam1)
         end
       ensure
         LibHTS.hts_itr_destroy(qiter)

data/lib/hts/bcf/format.rb

@@ -0,0 +1,52 @@
+# frozen_string_literal: true
+
+# https://github.com/brentp/hts-nim/blob/master/src/hts/vcf.nim
+# This is a port from Nim.
+# TODO: Make it more like Ruby.
+
+module HTS
+  class Bcf
+    class Format
+      def initialize(record)
+        @record = record
+        @p1 = FFI::MemoryPointer.new(:pointer) # FIXME: naming
+      end
+
+      def get(key, type = nil)
+        n = FFI::MemoryPointer.new(:int)
+        p1 = @p1
+        h = @record.bcf.header.struct
+        r = @record.struct
+
+        format_values = proc do |type|
+          ret = LibHTS.bcf_get_format_values(h, r, key, p1, n, type)
+          return nil if ret < 0 # return from method.
+
+          p1.read_pointer
+        end
+
+        case type.to_sym
+        when :int, :int32
+          format_values.call(LibHTS::BCF_HT_INT)
+                       .read_array_of_int32(n.read_int)
+        when :float, :real
+          format_values.call(LibHTS::BCF_HT_REAL)
+                       .read_array_of_float(n.read_int)
+        when :flag
+          format_values.call(LibHTS::BCF_HT_FLAG)
+                       .read_int == 1
+        when :string, :str
+          format_values.call(LibHTS::BCF_HT_STR)
+                       .read_pointer.read_string
+        end
+      end
+
+      def set; end
+
+      # def fields # iterator
+      # end
+
+      def genotypes; end
+    end
+  end
+end

data/lib/hts/bcf/header.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+module HTS
+  class Bcf
+    class Header
+      def initialize(h)
+        @h = h
+      end
+
+      def struct
+        @h
+      end
+
+      def to_ptr
+        @h.to_ptr
+      end
+    end
+  end
+end

data/lib/hts/bcf/info.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+module HTS
+  class Bcf
+    class Info
+      def initialize(record)
+        @record = record
+      end
+
+      def get(key, type = nil)
+        n = FFI::MemoryPointer.new(:int)
+        p1 = @record.p1
+        h = @record.bcf.header.struct
+        r = @record.struct
+
+        info_values = proc do |type|
+          ret = LibHTS.bcf_get_info_values(h, r, key, p1, n, type)
+          return nil if ret < 0 # return from method.
+
+          p1.read_pointer
+        end
+
+        case type.to_sym
+        when :int, :int32
+          info_values.call(LibHTS::BCF_HT_INT)
+                     .read_array_of_int32(n.read_int)
+        when :float, :real
+          info_values.call(LibHTS::BCF_HT_REAL)
+                     .read_array_of_float(n.read_int)
+        when :flag
+          info_values.call(LibHTS::BCF_HT_FLAG)
+                     .read_int == 1
+        when :string, :str
+          info_values.call(LibHTS::BCF_HT_STR)
+                     .read_pointer.read_string
+        end
+      end
+    end
+  end
+end