htslib 0.0.0 → 0.0.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8b19cf8cd36bbb9ffeb34a9c92b98adba85716f57d24ec6990cc95c53c8f658d
-  data.tar.gz: e678bdbe86be9c73c8fb3e378f86236a0a6eaf36ce047368af9263753bdca439
+  metadata.gz: c8048806df4c335ea698c8d3ad9e51e12644f0d32667d3db5fac59a004db75bf
+  data.tar.gz: 3d18183921f70b7b42dc5657195607c519c7d83c1e0e94b9b7a8f647d96f5504
 SHA512:
-  metadata.gz: cf6a74ee14a7f0d3bbff603c1af232d7a106a0c7d70e4c8e4c7ac4270ab98866a971611a793a1b6d994fde2b5d0a400b0e73fe04a99c9b2abfa16d5e0d82a2c5
-  data.tar.gz: d24f1fa238f3ad8d541f649a03cede9580bcdedad52e81a97f7d33a44a9c3d8fbfcf162b5912fcc9f7e5c832f09979d59cf7a4bc13fc6da7aa4facd0102fdfbb
+  metadata.gz: f50e96a23cb75130c5aa463329cce3c2d5784ed75c608d4685d2662ee22fa93e55208c356bce0fee319028c6ae8e33b5b20d270a17fb63ca6e39a736f668a2e4
+  data.tar.gz: c223f9b61979ace926fa53b408e85fa6f2e127cdde6ffa5bcfa1ac0ad7ce6c9c1ac5570522021c61e82d27498473299b8875f7d633b24367ae99dd2494758556

data/README.md

@@ -1,12 +1,26 @@
-# HTSlib
+# ruby-htslib
 
 [![Gem Version](https://badge.fury.io/rb/htslib.svg)](https://badge.fury.io/rb/htslib)
-![CI](https://github.com/kojix2/ruby-htslib/workflows/CI/badge.svg?branch=master)
+![CI](https://github.com/kojix2/ruby-htslib/workflows/CI/badge.svg)
 [![The MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE.txt)
+[![DOI](https://zenodo.org/badge/247078205.svg)](https://zenodo.org/badge/latestdoi/247078205)
+[![Docs Stable](https://img.shields.io/badge/docs-stable-blue.svg)](https://rubydoc.info/gems/htslib)
+
+:dna: [HTSlib](https://github.com/samtools/htslib) - for Ruby
+
+Ruby-htslib is the Ruby bindings to HTSlib, a C library for processing high throughput sequencing (HTS) data. 
+It will provide APIs to read and write file formats such as SAM, BAM, VCF, and BCF.
 
 :apple: Feel free to fork it out if you can develop it! 
 
-:bowtie: Just a prototype. 
+:bowtie: alpha stage.
+## Requirements
+
+* [Ruby](https://github.com/ruby/ruby) 2.7 or above.
+* [HTSlib](https://github.com/samtools/htslib)
+  * Ubuntu : `apt install libhts-dev`
+  * macOS : `brew install htslib`
+  * Build from source code (see Development section)
 
 ## Installation
 
@@ -14,34 +28,99 @@
 gem install htslib
 ```
 
-Set environment variable HTSLIBDIR. 
+If you have installed htslib with apt on Ubuntu or homebrew on Mac, [pkg-config](https://github.com/ruby-gnome/pkg-config) 
+will automatically detect the location of the shared library. 
+Alternatively, you can specify the directory of the shared library by setting the environment variable `HTSLIBDIR`.
 
 ```sh
-export HTSLIBDIR="/your/path/to/htslib"
+export HTSLIBDIR="/your/path/to/htslib" # libhts.so
 ```
 
-## Requirements
+## Overview
 
-* [htslib](https://github.com/samtools/htslib)
+### Low level API
 
-## Usage
+`HTS::LibHTS` provides native functions. 
 
 ```ruby
 require 'htslib'
 
-a = HTS::FFI.hts_open("a.bam", "r")
-b = HTS::FFI.hts_get_format(a)
+a = HTS::LibHTS.hts_open("a.bam", "r")
+b = HTS::LibHTS.hts_get_format(a)
 p b[:category]
 p b[:format]
 ```
 
+Note: Managed struct is not used in ruby-htslib. You may need to free the memory by yourself.
+
+### High level API (Plan)
+
+`Cram` `Bam` `Bcf` `Faidx` `Tabix`
+
+A high-level API is under development. We will change and improve the API to make it better.
+
+```ruby
+require 'htslib'
+
+bam = HTS::Bam.new("a.bam")
+
+bam.each do |r|
+  p name:  r.qname,
+    flag:  r.flag,
+    start: r.start + 1,
+    mpos:  r.mate_pos + 1,
+    mqual: r.mapping_quality,
+    seq:   r.sequence,
+    cigar: r.cigar.to_s,
+    qual:  r.base_qualities.map { |i| (i + 33).chr }.join
+end
+```
+
+## Documentation
+
+* [RubyDoc.info - HTSlib](https://rdoc.info/gems/htslib)
+
 ## Development
 
+To get started with development
+
+```sh
+git clone --recursive https://github.com/kojix2/ruby-htslib
+cd ruby-htslib
+bundle install
+bundle exec rake htslib:build
+bundle exec rake test
+```
+
+We plan to actively use the new features of Ruby. Since the number of users is small, backward compatibility is not important.
+On the other hand, we will consider compatibility with [Crystal](https://github.com/bio-crystal/htslib.cr) to some extent.
+
+#### FFI Extensions
+
+* [ffi-bitfield](https://github.com/kojix2/ffi-bitfield) : Extension of Ruby-FFI to support bitfields.
+
+#### Automatic generation or automatic validation (Future plan)
+
++ [c2ffi](https://github.com/rpav/c2ffi) is a tool to create JSON format metadata from C header files. It is planned to use c2ffi to automatically generate bindings or tests.
+
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/kojix2/ruby-htslib.
+Ruby-htslib is a library under development, so even small improvements like typofix are welcome! Please feel free to send us your pull requests.
+
+* [Report bugs](https://github.com/kojix2/ruby-htslib/issues)
+* Fix bugs and [submit pull requests](https://github.com/kojix2/ruby-htslib/pulls)
+* Write, clarify, or fix documentation
+* Suggest or add new features
+* [financial contributions](https://github.com/sponsors/kojix2)
+
+## Links
+
+* [samtools/hts-spec](https://github.com/samtools/hts-specs)
+* [bioruby](https://github.com/bioruby/bioruby)
 
+## Funding support
 
+This work was supported partially by [Ruby Association Grant 2020](https://www.ruby.or.jp/en/news/20201022).
 ## License
 
-The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+[MIT License](https://opensource.org/licenses/MIT).

data/lib/hts/bam/cigar.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+# Based on hts-python
+# https://github.com/quinlan-lab/hts-python
+
+module HTS
+  class Bam
+    class Cigar
+      include Enumerable
+
+      def initialize(pointer, n_cigar)
+        @pointer = pointer
+        @n_cigar = n_cigar
+      end
+
+      def to_ptr
+        @pointer
+      end
+
+      def to_s
+        to_a.flatten.join
+      end
+
+      def each
+        @n_cigar.times do |i|
+          c = @pointer[i].read_uint32
+          yield [LibHTS.bam_cigar_oplen(c),
+                 LibHTS.bam_cigar_opchr(c)]
+        end
+      end
+    end
+  end
+end

data/lib/hts/bam/flag.rb

@@ -0,0 +1,93 @@
+# frozen_string_literal: true
+
+# Based on hts-nim
+# https://github.com/brentp/hts-nim/blob/master/src/hts/bam/flag.nim
+
+module HTS
+  class Bam
+    class Flag
+      def initialize(flag_value)
+        raise TypeError unless flag_value.is_a? Integer
+
+        @value = flag_value
+      end
+
+      attr_accessor :value
+
+      # BAM_FPAIRED        =    1
+      # BAM_FPROPER_PAIR   =    2
+      # BAM_FUNMAP         =    4
+      # BAM_FMUNMAP        =    8
+      # BAM_FREVERSE       =   16
+      # BAM_FMREVERSE      =   32
+      # BAM_FREAD1         =   64
+      # BAM_FREAD2         =  128
+      # BAM_FSECONDARY     =  256
+      # BAM_FQCFAIL        =  512
+      # BAM_FDUP           = 1024
+      # BAM_FSUPPLEMENTARY = 2048
+
+      # TODO: Enabling bitwise operations
+      # hts-nim
+      # proc `and`*(f: Flag, o: uint16): uint16 {. borrow, inline .}
+      # proc `and`*(f: Flag, o: Flag): uint16 {. borrow, inline .}
+      # proc `or`*(f: Flag, o: uint16): uint16 {. borrow .}
+      # proc `or`*(o: uint16, f: Flag): uint16 {. borrow .}
+      # proc `==`*(f: Flag, o: Flag): bool {. borrow, inline .}
+      # proc `==`*(f: Flag, o: uint16): bool {. borrow, inline .}
+      # proc `==`*(o: uint16, f: Flag): bool {. borrow, inline .}
+
+      def paired?
+        has_flag? LibHTS::BAM_FPAIRED
+      end
+
+      def proper_pair?
+        has_flag? LibHTS::BAM_FPROPER_PAIR
+      end
+
+      def unmapped?
+        has_flag? LibHTS::BAM_FUNMAP
+      end
+
+      def mate_unmapped?
+        has_flag? LibHTS::BAM_FMUNMAP
+      end
+
+      def reverse?
+        has_flag? LibHTS::BAM_FREVERSE
+      end
+
+      def mate_reverse?
+        has_flag? LibHTS::BAM_FMREVERSE
+      end
+
+      def read1?
+        has_flag? LibHTS::BAM_FREAD1
+      end
+
+      def read2?
+        has_flag? LibHTS::BAM_FREAD2
+      end
+
+      def secondary?
+        has_flag? LibHTS::BAM_FSECONDARY
+      end
+
+      def qcfail?
+        has_flag? LibHTS::BAM_FQCFAIL
+      end
+
+      def dup?
+        has_flag? LibHTS::BAM_FDUP
+      end
+
+      def supplementary?
+        has_flag? LibHTS::BAM_FSUPPLEMENTARY
+      end
+
+      def has_flag?(o)
+        @value[o] != 0
+      end
+    end
+  end
+end

data/lib/hts/bam/header.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+# Based on hts-python
+# https://github.com/quinlan-lab/hts-python
+
+module HTS
+  class Bam
+    class Header
+      def initialize(pointer)
+        @sam_hdr = pointer
+      end
+
+      def struct
+        @sam_hdr
+      end
+
+      def to_ptr
+        @sam_hdr.to_ptr
+      end
+
+      # FIXME: better name?
+      def seqs
+        Array.new(@sam_hdr[:n_targets]) do |i|
+          LibHTS.sam_hdr_tid2name(@sam_hdr, i)
+        end
+      end
+
+      def text
+        LibHTS.sam_hdr_str(@sam_hdr)
+      end
+    end
+  end
+end

data/lib/hts/bam/record.rb

@@ -0,0 +1,176 @@
+# frozen_string_literal: true
+
+# Based on hts-python
+# https://github.com/quinlan-lab/hts-python
+
+module HTS
+  class Bam
+    class Record
+      SEQ_NT16_STR = "=ACMGRSVTWYHKDBN"
+
+      def initialize(bam1_t, header)
+        @bam1 = bam1_t
+        @header = header
+      end
+
+      def struct
+        @bam1
+      end
+
+      def to_ptr
+        @bam1.to_ptr
+      end
+
+      # def initialize_copy
+      #   super
+      # end
+
+      def self.rom_sam_str; end
+
+      def tags; end
+
+      # returns the query name.
+      def qname
+        LibHTS.bam_get_qname(@bam1).read_string
+      end
+
+      # Set (query) name.
+      # def qname=(name)
+      #   raise 'Not Implemented'
+      # end
+
+      # returns the tid of the record or -1 if not mapped.
+      def tid
+        @bam1[:core][:tid]
+      end
+
+      # returns the tid of the mate or -1 if not mapped.
+      def mate_tid
+        @bam1[:core][:mtid]
+      end
+
+      # returns 0-based start position.
+      def start
+        @bam1[:core][:pos]
+      end
+
+      # returns end position of the read.
+      def stop
+        LibHTS.bam_endpos @bam1
+      end
+
+      # returns 0-based mate position
+      def mate_start
+        @bam1[:core][:mpos]
+      end
+      alias mate_pos mate_start
+
+      # returns the chromosome or '' if not mapped.
+      def chrom
+        tid = @bam1[:core][:tid]
+        return "" if tid == -1
+
+        LibHTS.sam_hdr_tid2name(@header, tid)
+      end
+
+      # returns the chromosome of the mate or '' if not mapped.
+      def mate_chrom
+        tid = @bam1[:core][:mtid]
+        return "" if tid == -1
+
+        LibHTS.sam_hdr_tid2name(@header, tid)
+      end
+
+      def strand
+        LibHTS.bam_is_rev(@bam1) ? "-" : "+"
+      end
+
+      # def start=(v)
+      #   raise 'Not Implemented'
+      # end
+
+      # insert size
+      def isize
+        @bam1[:core][:isize]
+      end
+
+      # mapping quality
+      def mapping_quality
+        @bam1[:core][:qual]
+      end
+
+      # returns a `Cigar` object.
+      def cigar
+        Cigar.new(LibHTS.bam_get_cigar(@bam1), @bam1[:core][:n_cigar])
+      end
+
+      def qlen
+        LibHTS.bam_cigar2qlen(
+          @bam1[:core][:n_cigar],
+          LibHTS.bam_get_cigar(@bam1)
+        )
+      end
+
+      def rlen
+        LibHTS.bam_cigar2rlen(
+          @bam1[:core][:n_cigar],
+          LibHTS.bam_get_cigar(@bam1)
+        )
+      end
+
+      # return the read sequence
+      def sequence
+        r = LibHTS.bam_get_seq(@bam1)
+        seq = String.new
+        (@bam1[:core][:l_qseq]).times do |i|
+          seq << SEQ_NT16_STR[LibHTS.bam_seqi(r, i)]
+        end
+        seq
+      end
+
+      # return only the base of the requested index "i" of the query sequence.
+      def base_at(n)
+        n += @bam1[:core][:l_qseq] if n < 0
+        return "." if (n >= @bam1[:core][:l_qseq]) || (n < 0) # eg. base_at(-1000)
+
+        r = LibHTS.bam_get_seq(@bam1)
+        SEQ_NT16_STR[LibHTS.bam_seqi(r, n)]
+      end
+
+      # return the base qualities
+      def base_qualities
+        q_ptr = LibHTS.bam_get_qual(@bam1)
+        q_ptr.read_array_of_uint8(@bam1[:core][:l_qseq])
+      end
+
+      # return only the base quality of the requested index "i" of the query sequence.
+      def base_quality_at(n)
+        n += @bam1[:core][:l_qseq] if n < 0
+        return 0 if (n >= @bam1[:core][:l_qseq]) || (n < 0) # eg. base_quality_at(-1000)
+
+        q_ptr = LibHTS.bam_get_qual(@bam1)
+        q_ptr.get_uint8(n)
+      end
+
+      def flag_str
+        LibHTS.bam_flag2str(@bam1[:core][:flag])
+      end
+
+      # returns a `Flag` object.
+      def flag
+        Flag.new(@bam1[:core][:flag])
+      end
+
+      def to_s
+        kstr = LibHTS::KString.new
+        raise "Failed to format bam record" if LibHTS.sam_format1(@header.struct, @bam1, kstr) == -1
+
+        kstr[:s]
+      end
+
+      # TODO:
+      # def eql?
+      # def hash
+    end
+  end
+end

data/lib/hts/bam.rb

@@ -1,87 +1,114 @@
 # frozen_string_literal: true
 
-# Create a skeleton using hts-python as a reference.
+# Based on hts-python
 # https://github.com/quinlan-lab/hts-python
 
-class BamHeader
-  def initialize; end
-
-  def seqs; end
-end
-
-class Cigar
-  def initialize; end
-
-  def to_s; end
-
-  def inspect; end
-end
-
-class Alignment
-  def initialize; end
-
-  def self.rom_sam_str; end
-
-  def tags; end
-
-  def qname; end
-
-  def qname=; end
-
-  def rnext; end
-
-  def pnext; end
-
-  def rname; end
-
-  def strand; end
-
-  def base_qualities; end
-
-  def pos; end
-
-  def pos=; end
-
-  def isize; end
-
-  def mapping_quality; end
-
-  def cigar; end
-
-  def qlen; end
-
-  def rlen; end
-
-  def seqs; end
-
-  def flag_str; end
-
-  def flag; end
-
-  # def eql?
-  # def hash
-
-  def inspect; end
-
-  def to_s; end
-end
-
-class Bam
-  def initialize; end
-
-  def self.header_from_fasta; end
-
-  def inspect; end
-
-  def write; end
-
-  def close; end
-
-  def flush; end
-
-  def to_s; end
-
-  def each; end
-
-  # def call
+require_relative "bam/header"
+require_relative "bam/cigar"
+require_relative "bam/flag"
+require_relative "bam/record"
+require_relative "utils/open_method"
+
+module HTS
+  class Bam
+    include Enumerable
+    extend Utils::OpenMethod
+
+    attr_reader :file_path, :mode, :header
+    # HtfFile is FFI::BitStruct
+    attr_reader :htf_file
+
+    class << self
+      alias open new
+    end
+
+    def initialize(file_path, mode = "r", create_index: nil)
+      file_path = File.expand_path(file_path)
+
+      unless File.exist?(file_path)
+        message = "No such SAM/BAM file - #{file_path}"
+        raise message
+      end
+
+      @file_path = file_path
+      @mode      = mode
+      @htf_file  = LibHTS.hts_open(file_path, mode)
+      @header    = Bam::Header.new(LibHTS.sam_hdr_read(htf_file))
+
+      # FIXME: should be defined here?
+      @bam1      = LibHTS.bam_init1
+
+      # read
+      if mode[0] == "r"
+        # load index
+        @idx = LibHTS.sam_index_load(htf_file, file_path)
+        # create index
+        if create_index || (@idx.null? && create_index.nil?)
+          warn "Create index for #{file_path}"
+          LibHTS.sam_index_build(file_path, -1)
+          @idx = LibHTS.sam_index_load(htf_file, file_path)
+        end
+      else
+        # FIXME: implement
+        raise "not implemented yet."
+      end
+
+      # IO like API
+      if block_given?
+        begin
+          yield self
+        ensure
+          close
+        end
+      end
+    end
+
+    def struct
+      htf_file
+    end
+
+    def to_ptr
+      htf_file.to_ptr
+    end
+
+    def write(alns)
+      alns.each do
+        LibHTS.sam_write1(htf_file, header, alns.b) > 0 || raise
+      end
+    end
+
+    # Close the current file.
+    def close
+      LibHTS.hts_close(htf_file)
+    end
+
+    # Flush the current file.
+    def flush
+      # LibHTS.bgzf_flush(@htf_file.fp.bgzf)
+    end
+
+    def each(&block)
+      # Each does not always start at the beginning of the file.
+      # This is the common behavior of IO objects in Ruby.
+      # This may change in the future.
+      while LibHTS.sam_read1(htf_file, header, @bam1) > 0
+        record = Record.new(@bam1, header)
+        block.call(record)
+      end
+    end
+
+    # query [WIP]
+    def query(region)
+      qiter = LibHTS.sam_itr_querys(@idx, header, region)
+      begin
+        slen = LibHTS.sam_itr_next(htf_file, qiter, @bam1)
+        while slen > 0
+          yield Record.new(@bam1, header)
+          slen = LibHTS.sam_itr_next(htf_file, qiter, @bam1)
+        end
+      ensure
+        LibHTS.hts_itr_destroy(qiter)
+      end
+    end
+  end
 end