htslib 0.2.5 → 0.2.8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2a49c4758453c5c39915d9d17ccc33ce104699fca39c4083d4b2801767b655f1
-  data.tar.gz: 46ee49f6b452471e26afedfce70489cd24eebf86b1a18240bab125d465921fb8
+  metadata.gz: caaf2dd527c9570e2e4c72b22e5252a1ef69adbf983fbfe3e815c3a9bc1b91c0
+  data.tar.gz: b7e0b6ecf736142ea9b43e4ed7f80513b1c0d002ea40b2565e218be3a40d2fac
 SHA512:
-  metadata.gz: 918c0919750eb3d436cf19b1011ffed0a8d0760b29e0a3c2e675a9efa563a3f5c3b9a9ae38232e67192df4d066c446bbc0443f00d07cab9ac0c3401b38bd3fb7
-  data.tar.gz: 343fbaec89a2e60a54feadd29856b26b772baa65ef52d181cd5845ed53a8e549504ddcdcc739dff716f9069a5b0b3e6f22f5136c2a612cc6014eb79ee225ab2b
+  metadata.gz: 6fe725489c93915fc5afa5d58dd2a5a0030b82546f473b6cae4245898b01f60919494d64b31f3442c3ea316a25fb31c0b46cfdd447230c253c21818ca3da9fb8
+  data.tar.gz: f5bed585f7bded73022ea0ad9a1dc2eb6a10c13e63772cac267a3c23497b521d82d5ab82763e512b452d47f59be8c47dcf5ff2046c96dcc492604931799cf2b2

data/README.md

@@ -8,16 +8,16 @@
 
 Ruby-htslib is the [Ruby](https://www.ruby-lang.org) bindings to [HTSlib](https://github.com/samtools/htslib), a C library for high-throughput sequencing data formats. It allows you to read and write file formats commonly used in genomics, such as [SAM, BAM, VCF, and BCF](http://samtools.github.io/hts-specs/), in the Ruby language.
 
-:apple: Feel free to fork it! 
+:apple: Feel free to fork it!
 
 ## Requirements
 
-* [Ruby](https://github.com/ruby/ruby) 3.1 or above.
-* [HTSlib](https://github.com/samtools/htslib)
-  * Ubuntu : `apt install libhts-dev`
-  * macOS : `brew install htslib`
-  * Windows : [mingw-w64-htslib](https://packages.msys2.org/base/mingw-w64-htslib) is automatically fetched when installing the gem ([RubyInstaller](https://rubyinstaller.org) only).
-  * Build from source code (see Development section)
+- [Ruby](https://github.com/ruby/ruby) 3.1 or above.
+- [HTSlib](https://github.com/samtools/htslib)
+  - Ubuntu : `apt install libhts-dev`
+  - macOS : `brew install htslib`
+  - Windows : [mingw-w64-htslib](https://packages.msys2.org/base/mingw-w64-htslib) is automatically fetched when installing the gem ([RubyInstaller](https://rubyinstaller.org) only).
+  - Build from source code (see the Development section)
 
 ## Installation
 
@@ -25,21 +25,21 @@ Ruby-htslib is the [Ruby](https://www.ruby-lang.org) bindings to [HTSlib](https:
 gem install htslib
 ```
 
-If you have installed htslib with apt on Ubuntu or homebrew on Mac, [pkg-config](https://github.com/ruby-gnome/pkg-config) 
+If you have installed htslib with apt on Ubuntu or homebrew on Mac, [pkg-config](https://github.com/ruby-gnome/pkg-config)
 will automatically detect the location of the shared library. If pkg-config does not work well, set `PKG_CONFIG_PATH`.
 Alternatively, you can specify the directory of the shared library by setting the environment variable `HTSLIBDIR`.
 
 ```sh
-export HTSLIBDIR="/your/path/to/htslib" # libhts.so
+export HTSLIBDIR="/your/path/to/htslib" # Directory where libhts.so is located
 ```
 
-ruby-htslib also works on Windows; if you use RubyInstaller, htslib will be prepared automatically.
+ruby-htslib also works on Windows. If you use RubyInstaller, htslib will be prepared automatically.
 
-## Overview
+## Usage
 
-### High-level API
+### HTS::Bam - SAM / BAM / CRAM - Sequence Alignment Map file
 
-HTS::Bam - SAM / BAM / CRAM - Sequence Alignment Map file
+Reading fields
 
 ```ruby
 require 'htslib'
@@ -64,10 +64,24 @@ end
 bam.close
 ```
 
-HTS::Bcf - VCF / BCF - Variant Call Format file
+With a block
 
 ```ruby
-bcf = HTS::Bcf.open("b.bcf")
+HTS::Bam.open("test/fixtures/moo.bam") do |bam|
+  bam.each do |r|
+    puts r.to_s
+  end
+end
+```
+
+### HTS::Bcf - VCF / BCF - Variant Call Format file
+
+Reading fields
+
+```ruby
+require 'htslib'
+
+bcf = HTS::Bcf.open("test/fixtures/test.bcf")
 
 bcf.each do |r|
   p chrom:  r.chrom,
@@ -84,31 +98,38 @@ end
 bcf.close
 ```
 
-<details>
-<summary><b>Faidx</b></summary>
+With a block
 
 ```ruby
-fa = HTS::Faidx.open("c.fa")
+HTS::Bcf.open("test/fixtures/test.bcf") do |bcf|
+  bcf.each do |r|
+    puts r.to_s
+  end
+end
+```
 
-fa.fetch("chr1:1-10")
+### HTS::Faidx - FASTA / FASTQ - Nucleic acid sequence
 
+```ruby
+fa = HTS::Faidx.open("test/fixtures/moo.fa")
+fa.seq("chr1:1-10") # => CGCAACCCGA # 1-based
 fa.close
 ```
 
-</details>
-
-<details>
-<summary><b>Tbx</b></summary>
+### HTS::Tabix - GFF / BED - TAB-delimited genome position file
 
 ```ruby
-
+tb = HTS::Tabix.open("test/fixtures/test.vcf.gz")
+tb.query("poo", 2000, 3000) do |line|
+  puts line.join("\t")
+end
+tb.close
 ```
 
-</details>
-
 ### Low-level API
 
-`HTS::LibHTS` provides native C functions.
+Middle architectural layer between high-level Ruby code and low-level C code.
+`HTS::LibHTS` provides native C functions using [Ruby-FFI](https://github.com/ffi/ffi). 
 
 ```ruby
 require 'htslib'
@@ -119,25 +140,31 @@ p b[:category]
 p b[:format]
 ```
 
-Macro functions
+The low-level API makes it possible to perform detailed operations, such as calling CRAM-specific functions.
 
-htslib has a lot of macro functions for speed. Ruby-FFI cannot call C macro functions. However, essential functions are reimplemented in Ruby, and you can call them.
+#### Macro functions
 
-Structs
+HTSlib is designed to improve performance with many macro functions. However, it is not possible to call C macro functions directly from Ruby-FFI. To overcome this, important macro functions have been re-implemented in Ruby, allowing them to be called in the same way as native functions.
 
-Only a small number of C structs are implemented with FFI's ManagedStruct, which frees memory when Ruby's garbage collection fires. Other structs will need to be freed manually.
+#### Garbage Collection and Memory Freeing
+
+A small number of commonly used structs, such as `Bam1` and `Bcf1`, are implemented using FFI's `ManagedStruct`. This allows for automatic memory release when Ruby's garbage collection is triggered. On the other hand, other structs are implemented using `FFI::Struct`, and they will require manual memory release.
 
 ### Need more speed?
 
-Try Crystal. [HTS.cr](https://github.com/bio-cr/hts.cr) is implemented in Crystal language and provides an API compatible with ruby-htslib. Crystal language is not as flexible as Ruby language. You can not use `eval` methods and must always be careful with the types. Writing one-time scripts in Crystal may be less fun. However, if you have a clear idea of what you want to do in your mind, have already written code in Ruby, and need to run them over and over, try creating a command line tool in Crystal. The Crystal language is as fast as the Rust and C languages. It will give you incredible power to make tools.
+Try Crystal. [HTS.cr](https://github.com/bio-cr/hts.cr) is implemented in Crystal language and provides an API compatible with ruby-htslib. 
 
 ## Documentation
 
-* [API Documentation (develop branch)](https://kojix2.github.io/ruby-htslib/)
-* [RubyDoc.info - HTSlib](https://rdoc.info/gems/htslib)
+- [TUTORIAL.md](TUTORIAL.md)
+- [API Documentation (develop branch)](https://kojix2.github.io/ruby-htslib/)
+- [API Documentation (released gem)](https://rubydoc.info/gems/htslib)
 
 ## Development
 
+#### Compile from source code
+
+[GNU Autotools](https://en.wikipedia.org/wiki/GNU_Autotools) is required to compile htslib.
 To get started with development:
 
 ```sh
@@ -148,41 +175,60 @@ bundle exec rake htslib:build
 bundle exec rake test
 ```
 
-[GNU Autotools](https://en.wikipedia.org/wiki/GNU_Autotools) is required to compile htslib.
+#### Macro functions are reimplemented
 
 HTSlib has many macro functions. These macro functions cannot be called from FFI and must be reimplemented in Ruby.
 
-* Use the new version of Ruby to take full advantage of Ruby's potential.
-  * This is possible because we have a small number of users.
-* Remain compatible with [HTS.cr](https://github.com/bio-cr/hts.cr).
-  * The most challenging part is the return value. In the Crystal language, methods are expected to return only one type. On the other hand, in the Ruby language, methods that return multiple classes are very common. For example, in the Crystal language, the compiler gets confused if the return value is one of six types: Int32, Int64, Float32, Float64, Nil, or String. In fact Crystal allows you to do that. But the code gets a little messy. In Ruby, this is very common and doesn't cause any problems.
-  * Ruby and Crystal are languages that use garbage collection. However, the memory release policy for allocated C structures is slightly different: in Ruby-FFI, you can define a `self.release` method in `FFI::Struct`. This method is called when GC. So you don't have to worry about memory in high-level APIs like Bam::Record or Bcf::Record, etc. Crystal requires you to define a finalize method on each class. So you need to define it in Bam::Record or Bcf::Record.
+#### Use the latest Ruby
+
+Use Ruby 3 or newer to take advantage of new features. This is possible because we have a small number of users.
+
+#### Keep compatibility with Crystal language
+
+Compatibility with Crystal language is important for Ruby-htslib development. 
 
-Method naming generally follows the Rust-htslib API. 
+- [HTS.cr](https://github.com/bio-cr/hts.cr) - HTSlib bindings for Crystal
 
-#### FFI Extensions
+Return value
 
-* [ffi-bitfield](https://github.com/kojix2/ffi-bitfield) : Extension of Ruby-FFI to support bitfields.
+The most challenging part is the return value. In the Crystal language, methods are expected to return only one type. On the other hand, in the Ruby language, methods that return multiple classes are very common. For example, in the Crystal language, the compiler gets confused if the return value is one of six types: Int32, Int64, Float32, Float64, Nil, or String. In fact Crystal allows you to do that. But the code gets a little messy. In Ruby, this is very common and doesn't cause any problems.
+
+Memory management
+
+Ruby and Crystal are languages that use garbage collection. However, the memory release policy for allocated C structures is slightly different: in Ruby-FFI, you can define a `self.release` method in `FFI::Struct`. This method is called when GC. So you don't have to worry about memory in high-level APIs like Bam::Record or Bcf::Record, etc. Crystal requires you to define a finalize method on each class. So you need to define it in Bam::Record or Bcf::Record.
+
+Macro functions
+
+In ruby-htslib, C macro functions are added to `LibHTS`, but in Crystal, `LibHTS` is a Lib, so methods cannot be added. methods are added to `LibHTS2`.
+
+#### Naming convention
+
+If you are not sure about the naming of a method, follow the Rust-htslib API. This is a very weak rule. if a more appropriate name is found later in Ruby, it will replace it.
+
+#### Support for bitfields of structures
+
+Since Ruby-FFI does not support structure bit fields, the following extensions are used.
+
+- [ffi-bitfield](https://github.com/kojix2/ffi-bitfield) - Extension of Ruby-FFI to support bitfields.
 
 #### Automatic validation
 
 In the `script` directory, there are several tools to help implement ruby-htslib. Scripts using c2ffi can check the coverage of htslib functions in Ruby-htslib. They are useful when new versions of htslib are released.
 
-* [c2ffi](https://github.com/rpav/c2ffi) is a tool to create JSON format metadata from C header files. 
+- [c2ffi](https://github.com/rpav/c2ffi) is a tool to create JSON format metadata from C header files.
 
 ## Contributing
 
 Ruby-htslib is a library under development, so even minor improvements like typo fixes are welcome! Please feel free to send us your pull requests.
 
-* [Report bugs](https://github.com/kojix2/ruby-htslib/issues)
-* Fix bugs and [submit pull requests](https://github.com/kojix2/ruby-htslib/pulls)
-* Write, clarify, or fix documentation
-* Suggest or add new features
-* [financial contributions](https://github.com/sponsors/kojix2)
+- [Report bugs](https://github.com/kojix2/ruby-htslib/issues)
+- Fix bugs and [submit pull requests](https://github.com/kojix2/ruby-htslib/pulls)
+- Write, clarify, or fix documentation
+- Suggest or add new features
+- [financial contributions](https://github.com/sponsors/kojix2)
 
-
-```
-# Ownership and Commitment Rights
+```markdown
+# Ownership and Commit Rights
 
 Do you need commit rights to the ruby-htslib repository?
 Do you want to get admin rights and take over the project?
@@ -195,8 +241,8 @@ One of the greatest joys of using a minor language like Ruby in bioinformatics i
 
 ## Links
 
-* [samtools/hts-spec](https://github.com/samtools/hts-specs)
-* [bioruby](https://github.com/bioruby/bioruby)
+- [samtools/hts-spec](https://github.com/samtools/hts-specs)
+- [bioruby](https://github.com/bioruby/bioruby)
 
 ## Funding support
 

data/TUTORIAL.md

@@ -0,0 +1,316 @@
+# Tutorial
+
+```mermaid
+%%{init:{'theme':'base'}}%%
+classDiagram
+Bam~Hts~ o-- `Bam::Header`
+Bam o-- `Bam::Record`
+`Bam::Record` o-- `Bam::Header`
+Bcf~Hts~ o-- `Bcf::Header`
+Bcf o-- `Bcf::Record`
+`Bcf::Record` o--`Bcf::Header`
+`Bam::Header` o-- `Bam::HeaderRecord`
+`Bcf::Header` o-- `Bcf::HeaderRecord`
+`Bam::Record` o-- Flag
+`Bam::Record` o-- Cigar
+`Bam::Record` o-- Aux
+`Bcf::Record` o-- Info
+`Bcf::Record` o-- Format
+class Bam{
+  +@hts_file : FFI::Struct
+  +@header : Bam::Header
+  +@file_name
+  +@index_name
+  +@mode
+  +struct()
+  +build_index()
+  +each() Enumerable
+  +query()
+}
+class Bcf{
+  +@hts_file : FFI::Struct
+  +@header : Bcf::Header
+  +@file_name
+  +@index_name
+  +@mode
+  +struct()
+  +build_index()
+  +each() Enumerable
+  +query()
+}
+class Tabix~Hts~{
+  +@hts_file : FFI::Struct
+}
+class `Bam::Header`{
+  +@sam_hdr : FFI::Struct
+  +struct()
+  +target_count()
+  +target_names()
+  +name2tid()
+  +tid2name()
+  +to_s()
+}
+class `Bam::Record` {
+  +@bam1 : FFI::Struct
+  +@header : Bam::Header
+  +struct()
+  +qname()
+  +qname=()
+  +tid()
+  +tid=()
+  +mtid()
+  +mtid=()
+  +pos()
+  +pos=()
+  +mpos() +mate_pos()
+  +mpos=() +mate_pos=()
+  +bin()
+  +bin=()
+  +endpos()
+  +chorm() +contig()
+  +mate_chrom() +mate_contig()
+  +strand()
+  +mate_strand()
+  +isize() +insert_size()
+  +isize=() +insert_size=()
+  +mapq()
+  +mapq=()
+  +cigar()
+  +qlen()
+  +rlen()
+  +seq() +sequence()
+  +len()
+  +base(n)
+  +qual()
+  +qual_string()
+  +base_qual(n)
+  +flag()
+  +flag=()
+  +aux()
+  +to_s()
+}
+class `Aux` {
+  -@record : Bam::Record
+  +[]()
+  +get_int()
+  +get_float()
+  +get_string()
+}
+class `Bcf::Header`{
+  +@bcf_hdr : FFI::Struct
+  +struct()
+  +to_s()
+}
+class `Bcf::Record`{
+  +@bcf1 : FFI::Struct
+  +@header : Bcf::Header
+  +struct()
+  +rid()
+  +rid=()
+  +chrom()
+  +pos()
+  +pos=()
+  +id()
+  +id=()
+  +clear_id()
+  +ref()
+  +alt()
+  +alleles()
+  +qual()
+  +qual=()
+  +filter()
+  +info()
+  +format()
+  +to_s()
+}
+class Flag {
+  +@value : Integer
+  +paired?()
+  +proper_pair?()
+  +unmapped?()
+  +mate_unmapped?()
+  +reverse?()
+  +mate_reverse?()
+  +read1?()
+  +read2?()
+  +secondary?()
+  +qcfail?()
+  +duplicate?()
+  +supplementary?()
+  +&()
+  +|()
+  +^()
+  +~()
+  +<<()
+  +>>()
+  +to_i()
+  +to_s()
+}
+class Info {
+  -@record : Bcf::Record
+  +[]()
+  +get_int()
+  +get_float()
+  +get_string()
+  +get_flag()
+  +fields()
+  +length() +size()
+  +to_h()
+  -info_ptr()
+}
+class Format {
+  -@record : Bcf::Record
+  +[]()\
+  +get_int()
+  +get_float()
+  +get_string()
+  +get_flag()
+  +fields()
+  +length() +size()
+  +to_h()
+  -format_ptr()
+}
+class Cigar {
+  -@array : Array
+  +each() Enumerable
+  +qlen()
+  +rlen()
+  +to_s()
+  +==()
+  +eql?()
+}
+class Faidx{
+  +@fai
+}
+
+```
+
+## Installation
+
+```
+gem install htslib
+```
+
+You can check which shared libraries are used by ruby-htslib as follows
+
+```ruby
+require "htslib"
+puts HTS.lib_path
+# => "/home/kojix2/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/htslib-0.2.6/vendor/libhts.so"
+```
+
+## HTS::Bam - SAM / BAM / CRAM - Sequence Alignment Map file
+
+Reading fields
+
+```ruby
+require 'htslib'
+
+bam = HTS::Bam.open("test/fixtures/moo.bam")
+
+bam.each do |r|
+  pp name: r.qname,
+     flag: r.flag,
+     chrm: r.chrom,
+     strt: r.pos + 1,
+     mapq: r.mapq,
+     cigr: r.cigar.to_s,
+     mchr: r.mate_chrom,
+     mpos: r.mpos + 1,
+     isiz: r.isize,
+     seqs: r.seq,
+     qual: r.qual_string,
+     MC:   r.aux("MC")
+end
+
+bam.close
+```
+
+Open with block
+
+```ruby
+HTS::Bam.open("test/fixtures/moo.bam") do |bam|
+  bam.each do |record|
+    # ...
+  end
+end
+```
+
+Writing
+
+```ruby
+in = HTS::Bam.open("foo.bam")
+out = HTS::Bam.open("bar.bam", "wb")
+
+out.header = in.header
+# out.write_header(in.header)
+
+in.each do |r|
+  out << r
+  # out.write(r)
+end
+
+in.close
+out.close
+```
+
+Create index
+
+```ruby
+HTS::Bam.open("foo.bam", build_index: true)
+```
+
+```
+b = HTS::Bam.open("foo.bam")
+            .build_index
+            .load_index
+```
+
+## HTS::Bcf - VCF / BCF - Variant Call Format file
+
+Reading fields
+
+```ruby
+bcf = HTS::Bcf.open("b.bcf")
+
+bcf.each do |r|
+  p chrom:  r.chrom,
+    pos:    r.pos,
+    id:     r.id,
+    qual:   r.qual.round(2),
+    ref:    r.ref,
+    alt:    r.alt,
+    filter: r.filter,
+    info:   r.info.to_h,
+    format: r.format.to_h
+end
+
+bcf.close
+```
+
+Open with block
+
+```ruby
+HTS::Bcf.open("b.bcf") do |bcf|
+  bcf.each do |record|
+    # ...
+  end
+end
+```
+
+Writing
+
+```ruby
+in = HTS::Bcf.open("foo.bcf")
+out = HTS::Bcf.open("bar.bcf", "wb")
+
+out.header = in.header
+# out.write_header(in.header)
+in.each do |r|
+  out << r
+  # out.write(r)
+end
+
+in.close
+out.close
+```