bioinform 0.1.17 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 792006e928db4e7ce443f56f93b94e9f5cbc2e95
-  data.tar.gz: f7f4f25e156071fee5f242cc50ecdcd80741b596
+  metadata.gz: 41bb8dd19247a6f1b8e7643e5fbf1d0e03b823de
+  data.tar.gz: 7dbd3f01dbea7fe1ed3125bef775cc72e5dccf8e
 SHA512:
-  metadata.gz: 4d0c1b14cba03745ecdf19b14579533489619f22f584e3fda64a77b4d72ba0998930b8239ceb5212b0d09ca6aa88879abc29057354cd8b4650ae35dab5845a7e
-  data.tar.gz: 8e0a3b680e4e5ef765697abd6c0c449dc303ea75c1d886aef4c01a4a34f38a67008ac9bf2dc4f173ba0f7a3cd769cd22424670689fa727946bcc15767a5689ed
+  metadata.gz: a3e5c829bf134e07c7a03a56de61017ca9f3c7237e1c6a18e49d0cf57cdf1a3054c7501541070e85a756ae12231a84c6d04a6f94384b5d7dd57f93ceaf335e11
+  data.tar.gz: 84d64628b85fd7b5e757637de51c074baeb98ebd8053c83c4af4b77bcf7e843fca10e545b472e7fda781cc9b3fba4ccb9b8bd8055e254b5f7a1f83d1575f672f

data/Gemfile

@@ -6,11 +6,11 @@ gemspec
 
 group :development do
 #  gem 'win32console'
-  gem 'rspec', '>= 2.0'
-  gem 'fabrication', '~> 2.5.0'
+  gem 'rspec', '~> 3.0'
+  # gem 'fabrication', '~> 2.5.0'
   gem 'rspec-given', '>= 2.0.0'
   gem 'spork', '>= 0.9.2'
-  gem 'fakefs', '~> 0.4.2'
+  gem 'fakefs', '~> 0.4.2', :require => 'fakefs/safe'
   gem 'wdm', :require => false
   gem 'guard-rspec', '>=2.1.0'
 end

data/LICENSE

@@ -1,5 +1,4 @@
 Copyright (c) 2012-2014 Ilya Vorontsov
-bioinform utilizes several methods from the `activesupport` gem.
 
 MIT License
 

data/README.md

@@ -19,7 +19,7 @@ Or install it yourself as:
 
 ## Usage
 
-Usage is under construction. I don't recommend use this gem for a while: syntax is on the way to change to more simple and concise. But stay tuned
+Usage is under construction. I don't recommend to use this gem for a while: syntax is on the way to change to more simple and concise. But stay tuned
 
 ### Command-line applications
   * pcm2pwm

data/TODO.txt

@@ -1,38 +1,31 @@
-! Make matrices immutable  - it will allow more safe interface and better caching 
-
-Collection contain Motif-s, each Motif can contain any of list: pcm,pwm,ppm.
-Name, background, tags and any other parameters should be removed from PM class to be placed in Motif
-
-
-
-ToDo:
-  how to make PM#equal? and PM#hash so that using PMs in Sets wouldn't destroy comparability of Sets and two sets with the same PMs(but different objects) would be equal. (also using pm as a hash-key)
-  Make specs and fix code in such a way that Parser.split_on_motifs and so on returned consistent result. E.g. Parser.parse! raised an error on multiple times invocation
-
-  refactor CLI::SplitMotifs in place where it splits collection file and choose real data models or makes PM
-  
-  Make Collection convenient way to store both pwm and pcm for a single motif (may be both should be in parameters of motif?). Also make methods like sort! that can change collection inner structure without working with @collection-variable directly. For example collection.sort!{|a,b| a.length<=>b.length} (here sort yielded only motif, but now it yields both motif and infos - it's inconvenient)
-
-Make parser exception print out text where parsing was broken (processing line +- 2 nearest lines and command and line numbers)
-Prevent parser going into infinity loop
+сделать работу с ValidationError
+сделать ошибки тэгированными
+обобщить модели фона на разные алфавиты
+парсеры
+  подумать про большее число парсеров: transfac etc
+  должны ли парсеры быть в библиотеке или снаружи
+  не стоит ли парсеры утащить в MotifModel или еще куда-нибудь? У нас ведь еще будут парсеры сиквенсов итп
+форматтеры
+  должны ли форматтеры быть в библиотеке или снаружи
+конвертеры
+  конвертер мары должен быть вынесен отдельно
+  конвертеры хорошо быть отрефакторить (но не ясно как это сделать хорошо)
+  посмотреть, совместимы ли конвертеры с идеей разных алфавитов
+включить модели сиквенсов (оптимизировать их: нуклекотид-число; не забыть про разные алфавиты)/снипов/алигнментов/геномных позиций-интервалов
+скоринг IUPAC-сиквенсов сейчас делается при помощи IUPAC-алфавитных матриц (см. конвертер PWM2IupacPWM). Написать хелпы.
+утащить CLI из пакета куда-нибудь
+утащить из bioinform.rb get_pcm
+починить convert_motif
+
+PM#equal? и PM#hash
+
+? Make parser exception print out text where parsing was broken (processing line +- 2 nearest lines and command and line numbers)
 
 Create CLI-apps:
--- to merge many files(or whole folder) to a Collection (in a way that makes able to give collection a name)
-
-Make Parsers to be switcheable in runtime so that one could parse string composed of two motifs in different formats.
+? -- to merge many files(or whole folder) to a Collection (in a way that makes able to give collection a name)
 
 Decide:
--- Whether PPM should have `words_count`/`weight`?
-   PPM format such that parser got both matrix and count (if PPM have `word_count`)
--- can_parse?
 -- Whether to cache suffices:  cache :best_suffix, obsolete: [:discrete!, :background!, ...]
--- behaviour of PM#== for PMs with different tags
--- should background be in PM by default?
--- refactor PM.new #== and so on to make possible consistently introduce or remove a variable at a single line
--- Make PCM#valid? and PPM#valid? more specific. This shouldn't destroy functionality to load arbitrary data as matrix, but only in force mode (I don't understand yet where should it be: in a constructor or where? And which validation-"severity" levels should be? Strong validation - size-only-validation - size-and-type-validation - no validation ???  or may be  options: valid_strictness: 'strict', 'usual', 'strict_with_name' ??? It should be considered)
--- PM#to_pcm and friends have unintuitive behavior. E.g. pm.to_pcm.to_pwm != pm.to_pwm  First is matrix treated as pcm and then converted, while second is matrix treated as pwm from start
--- Should parser be reloadable or not? May be delete #reset_scanner?
--- Should Collection has infos for each motif if it already has parameters? (see also discussion above about Collection#sort! and so on)
 
 Specs
 -- PWM#probabilities, #score_variance, #gauss_estimation

data/bin/convert_motif

@@ -0,0 +1,4 @@
+#!/usr/bin/env ruby
+
+require_relative '../lib/bioinform/cli/convert_motif'
+Bioinform::CLI::ConvertMotif.main(ARGV)

data/bin/pcm2pwm

@@ -1,4 +1,4 @@
 #!/usr/bin/env ruby
 
 require_relative '../lib/bioinform/cli/pcm2pwm'
-Bioinform::CLI::PCM2PWM.main(ARGV)
+Bioinform::CLI::PCM2PWM.main(ARGV)

data/bin/split_motifs

@@ -1,4 +1,4 @@
 #!/usr/bin/env ruby
 
 require_relative '../lib/bioinform/cli/split_motifs'
-Bioinform::CLI::SplitMotifs.main(ARGV)
+Bioinform::CLI::SplitMotifs.main(ARGV)

data/bioinform.gemspec

@@ -14,6 +14,4 @@ Gem::Specification.new do |gem|
   gem.name          = "bioinform"
   gem.require_paths = ["lib"]
   gem.version       = Bioinform::VERSION
-
-  gem.add_dependency('docopt', '= 0.5.0')
 end

data/lib/bioinform.rb

@@ -1,37 +1,75 @@
 require_relative 'bioinform/version'
 require_relative 'bioinform/support'
+require_relative 'bioinform/errors'
 require_relative 'bioinform/parsers'
-require_relative 'bioinform/formatters'
 require_relative 'bioinform/data_models'
+require_relative 'bioinform/conversion_algorithms'
+require_relative 'bioinform/formatters'
 require_relative 'bioinform/cli'
 
+require_relative 'bioinform/background'
+require_relative 'bioinform/alphabet'
+
 module Bioinform
-  class Error < StandardError
+  def self.get_model(data_model, matrix, name)
+    Bioinform::MotifModel.const_get(data_model).new(matrix).named(name)
+  end
+
+  def self.get_model_from_string(data_model, matrix_string)
+    motif_infos = MatrixParser.new.parse(matrix_string)
+    get_model(data_model, motif_infos.matrix, name)
   end
 
   def self.get_pwm(data_model, matrix, background, pseudocount, effective_count)
-    pm = Bioinform.const_get(data_model).new(matrix)
-    pm.set_parameters(background: background)
-    if pseudocount && ! pseudocount.blank? && [:PCM,:PPM].include?(data_model.to_sym)
-      pm.set_parameters(pseudocount: pseudocount)
+    input_model = get_model_from_string(data_model, matrix)
+    case input_model
+    when MotifModel::PPM
+      ppm2pcm_converter = ConversionAlgorithms::PPM2PCM.new(count: effective_count)
+      pcm2pwm_converter = ConversionAlgorithms::PCM2PWM.new(background: background, pseudocount: pseudocount)
+      pcm2pwm_converter.convert(ppm2pcm_converter.convert(input_model))
+    when MotifModel::PCM
+      pcm2pwm_converter = ConversionAlgorithms::PCM2PWM.new(background: background, pseudocount: pseudocount)
+      pcm2pwm_converter.convert(input_model)
+    when MotifModel::PWM
+      input_model
+    else
+      raise Error, "Unknown input `#{input_model}`"
     end
-    if effective_count && [:PPM].include?(data_model.to_sym)
-      pm.set_parameters(effective_count: effective_count)
-    end
-    pm.to_pwm
   rescue => e
-    raise "PWM creation failed (#{e})"
+    raise Error, "PWM creation failed (#{e})"
   end
 
   def self.get_pcm(data_model, matrix, effective_count)
-    pm = Bioinform.const_get(data_model).new(matrix)
-    if effective_count && [:PPM].include?(data_model.to_sym)
-      pm.set_parameters(effective_count: effective_count)
+    input_model = get_model_from_string(data_model, matrix)
+    case input_model
+    when MotifModel::PPM
+      ppm2pcm_converter = ConversionAlgorithms::PPM2PCM.new(count: effective_count)
+      ppm2pcm_converter.convert(input_model)
+    when MotifModel::PCM
+      input_model
+    when MotifModel::PWM
+      raise Error, 'Conversion PWM-->PCM not yet implemented'
+    else
+      raise Error, "Unknown input `#{input_model}`"
     end
-    pm.to_pcm
+  rescue => e
+    raise Error, "PCM creation failed (#{e})"
   end
 
   def self.get_ppm(data_model, matrix)
-    Bioinform.const_get(data_model).new(matrix).to_ppm
+    input_model = get_model_from_string(data_model, matrix)
+    case input_model
+    when MotifModel::PPM
+      input_model
+    when MotifModel::PCM
+      pcm2ppm_converter = ConversionAlgorithms::PCM2PPM.new
+      pcm2ppm_converter.convert(input_model)
+    when MotifModel::PWM
+      raise Error, 'Conversion PWM-->PPM not yet implemented'
+    else
+      raise Error, "Unknown input `#{input_model}`"
+    end
+  rescue => e
+    raise Error, "PPM creation failed (#{e})"
   end
 end

data/lib/bioinform/alphabet.rb

@@ -0,0 +1,85 @@
+require_relative 'support'
+require_relative 'errors'
+
+module Bioinform
+  # alphabets for DNA/RNA (which do have complements)
+  class ComplementableAlphabet
+    attr_reader :alphabet, :complement_alphabet
+
+    # ComplementableAlphabet.new([:A,:C,:G,:T], [:T,:G,:C,:A])
+    def initialize(alphabet, complements)
+      @alphabet = alphabet.map{|letter| letter.upcase.to_sym }
+      @complement_alphabet = complements.map{|letter| letter.upcase.to_sym }
+
+      @complements_by_letters = Support.various_key_value_case_types( Hash[ @alphabet.zip(@complement_alphabet) ] )
+
+      @index_by_letter = Support.various_key_case_types(Support.element_indices(@alphabet))
+      raise Error, "Complement's complement should be original letter"  unless valid?
+    end
+
+    def valid?
+      non_duplicated_letters = (@alphabet.size == @alphabet.uniq.size)
+      compatible_sizes = (@alphabet.size == @complement_alphabet.size)
+      invertable_complement = @alphabet.all?{|letter| complement_letter(complement_letter(letter)) == letter }
+      non_duplicated_letters && compatible_sizes && invertable_complement
+    end
+    private :valid?
+
+    def size
+      @alphabet.size
+    end
+
+    def each_letter(&block)
+      @alphabet.each(&block)
+    end
+
+    def each_letter_index(&block)
+      @alphabet.each_index(&block)
+    end
+
+    def letter_by_index(index)
+      @alphabet[index] || raise(Error, "Unknown letter-index #{index}")
+    end
+
+    def index_by_letter(letter)
+      @index_by_letter[letter] || raise(Error, "Unknown letter #{letter}")
+    end
+
+    def complement_letter(letter)
+      @complements_by_letters[letter] || raise(Error, "Unknown letter #{letter}")
+    end
+
+    def complement_index(index)
+      letter = @complement_alphabet[index] || raise(Error, "Unknown letter-index #{index}")
+      @index_by_letter[letter]
+    end
+
+    def ==(other)
+      @alphabet == other.alphabet && @complement_alphabet == other.complement_alphabet
+    end
+  end
+
+
+  module IUPAC
+    NucleotideIndicesByIUPACLetter = {
+      A: [0], C: [1], G: [2], T: [3],
+      M: [0, 1], R: [0, 2], W: [0, 3], S: [1, 2], Y: [1, 3], K: [2, 3],
+      V: [0, 1, 2], H: [0, 1, 3], D: [0, 2, 3], B: [1, 2, 3],
+      N: [0, 1, 2, 3]
+    }
+    IUPACLettersByNucleotideIndices = Bioinform::Support.with_key_permutations(NucleotideIndicesByIUPACLetter.invert)
+
+    def self.complement_iupac_letter(iupac_letter)
+      nucleotide_indices = NucleotideIndicesByIUPACLetter[iupac_letter]
+      complement_nucleotide_indices = nucleotide_indices.map{|nucleotide_index| 3 - nucleotide_index }
+      IUPACLettersByNucleotideIndices[complement_nucleotide_indices]
+    end
+  end
+
+  iupac_letters = [:A, :C, :G, :T, :M, :R, :W, :S, :Y, :K, :V, :H, :D, :B, :N]
+
+  NucleotideAlphabet = ComplementableAlphabet.new([:A,:C,:G,:T], [:T,:G,:C,:A])
+  NucleotideAlphabetWithN = ComplementableAlphabet.new([:A,:C,:G,:T,:N], [:T,:G,:C,:A,:N])
+  IUPACAlphabet = ComplementableAlphabet.new( iupac_letters,
+                                              iupac_letters.map{|letter| IUPAC.complement_iupac_letter(letter) } )
+end

data/lib/bioinform/background.rb

@@ -0,0 +1,90 @@
+require_relative 'errors'
+
+# TODO: generalize for the case of different alphabet
+module Bioinform
+  # it also tags Frequencies and WordwiseBackground classes so that .is_a?(Bioinform::Background) is true for them
+  module Background
+    def self.wordwise
+      Bioinform::Background::Wordwise
+    end
+    def self.uniform
+      Bioinform::Background::Uniform
+    end
+
+    def self.from_gc_content(gc_content)
+      p_at = (1.0 - gc_content) / 2.0;
+      p_cg = gc_content / 2.0;
+      Frequencies.new([p_at, p_cg, p_cg, p_at])
+    end
+
+    def self.from_string(str)
+      return wordwise  if str.downcase == 'wordwise'
+      return uniform  if str.downcase == 'uniform'
+      arr = str.strip.split(',').map(&:to_f)
+      arr == [1,1,1,1] ? wordwise : Bioinform::Frequencies.new(arr)
+    end
+  end
+
+  module FrequencyCalculations
+    # sum(values_i * p_i)
+    def mean(values)
+      4.times.map{|i| values[i] * frequencies[i] }.inject(0.0, &:+)
+    end
+    # sum(values_i^2 * p_i)
+    def mean_square(values)
+      4.times.map{|i| values[i] * values[i] * frequencies[i] }.inject(0.0, &:+)
+    end
+
+    def symmetric?
+      frequencies == frequencies.reverse
+    end
+  end
+
+  class Frequencies
+    include FrequencyCalculations
+    include Bioinform::Background
+    def initialize(frequencies)
+      @frequencies = frequencies
+      raise Error, 'Sum of Background frequencies should be equal to 1' unless (frequencies.inject(0.0, &:+) - 1.0).abs < 1e-4
+    end
+
+    attr_reader :frequencies
+    def counts; frequencies; end
+    def volume; 1; end
+    def wordwise?; false; end
+
+
+    def ==(other)
+      self.class == other.class && frequencies == other.frequencies
+    end
+
+    def to_s
+      counts.join(',')
+    end
+  end
+
+  class WordwiseBackground
+    UniformFrequencies = [0.25, 0.25, 0.25, 0.25]
+    WordwiseCounts = [1, 1, 1, 1]
+    include FrequencyCalculations
+    include Bioinform::Background
+
+    def frequencies; UniformFrequencies; end
+    def counts; WordwiseCounts; end
+    def volume; 4; end
+    def wordwise?; true; end
+
+    def ==(other)
+      self.class == other.class
+    end
+
+    def to_s
+      counts.join(',')
+    end
+  end
+
+  module Background
+    Uniform = Bioinform::Frequencies.new([0.25, 0.25, 0.25, 0.25])
+    Wordwise = Bioinform::WordwiseBackground.new
+  end
+end

data/lib/bioinform/cli.rb

@@ -25,6 +25,5 @@ module Bioinform
   end
 end
 
-require_relative 'cli/merge_into_collection'
 require_relative 'cli/pcm2pwm'
-require_relative 'cli/split_motifs'
+require_relative 'cli/split_motifs'

data/lib/bioinform/cli/convert_motif.rb

@@ -7,14 +7,14 @@ $logger = Logger.new('convert_motif.log')
 module Bioinform
   module CLI
     class ConvertMotif
-      
+
       def arguments
        @arguments ||= []
       end
       def options
         @options ||= {}
       end
-      
+
       def main(argv)
         parse!(argv, filename_format: './{name}.{ext}')
         motif_files = arguments
@@ -23,28 +23,60 @@ module Bioinform
           puts option_parser.help()
           return
         end
-        
+
         output_motifs = []
         motifs = motif_files.map do |filename|
+          input = File.read(filename)
+          motif_info = MotifParser.new.parse(input)
           case options[:model_from]
           when 'pwm'
-            PWM.new(File.read(filename))
+            MotifModel::PWM.new(motif_info[:matrix]).named(motif_info[:name])
           when 'pcm'
-            PCM.new(File.read(filename))
+            MotifModel::PCM.new(motif_info[:matrix]).named(motif_info[:name])
           when 'ppm'
-            PPM.new(File.read(filename))
+            MotifModel::PPM.new(motif_info[:matrix]).named(motif_info[:name])
+          else
+            raise "Unknown value of model-from parameter: `#{options[:model_from]}`"
           end
         end
-        
+        pcm2pwm_converter = ConversionAlgorithms::PCM2PWMConverter.new(pseudocount: :log, background: Background::Uniform)
+        pcm2ppm_converter = ConversionAlgorithms::PCM2PPMConverter.new
+        ppm2pcm_converter = ConversionAlgorithms::PPM2PCMConverter.new(count: 100)
         motifs.each do |motif|
           begin
             case options[:model_to]
             when 'pwm'
-              output_motifs << motif.to_pwm
+              if MotifModel.acts_as_pcm?(motif)
+                output_motifs << pcm2pwm_converter.convert(motif)
+              elsif MotifModel.acts_as_ppm?(motif)
+                output_motifs << pcm2pwm_converter.convert(ppm2pcm_converter.convert(motif))
+              elsif MotifModel.acts_as_pwm?(motif)
+                output_motifs << motif
+              else
+                raise "Can't be here"
+              end
             when 'pcm'
-              output_motifs << motif.to_pcm
+              if MotifModel.acts_as_pcm?(motif)
+                output_motifs << motif
+              elsif MotifModel.acts_as_ppm?(motif)
+                output_motifs << ppm2pcm_converter.convert(motif)
+              elsif MotifModel.acts_as_pwm?(motif)
+                raise 'Not yet implemented'
+              else
+                raise "Can't be here"
+              end
             when 'ppm'
-              output_motifs << motif.to_ppm
+              if MotifModel.acts_as_pcm?(motif)
+                output_motifs << pcm2ppm_converter.convert(motif)
+              elsif MotifModel.acts_as_ppm?(motif)
+                output_motifs << motif
+              elsif MotifModel.acts_as_pwm?(motif)
+                raise 'Not yet implemented'
+              else
+                raise "Can't be here"
+              end
+            else
+              raise "Unknown value of model-to parameter: `#{options[:model_to]}`"
             end
           rescue
             $stderr.puts "One can't convert from #{options[:model_from]} data-model to #{options[:model_to]} data-model"
@@ -52,8 +84,10 @@ module Bioinform
           end
         end
         puts output_motifs.join("\n\n")
-      rescue
+      rescue => e
         $stderr.puts "Error! Conversion wasn't performed"
+        $stderr.puts e
+        $stderr.puts e.backtrace
       end
 
       def option_parser
@@ -62,13 +96,14 @@ module Bioinform
             Usage:
               convert_motif [options] <motif-files>...
               ls | convert_motif [options]
-              
+
             convert_motif - tool for converting motifs from different input formats
                                                        to different output formats.
             It can change both formatting style and motif models.
             Resulting model is sent to stdout (this can be overriden with --save option).
           BANNER
-          
+
+          cli.version = ::Bioinform::VERSION
           cli.summary_indent = ''
           cli.banner = strip_doc(banner)
           cli.separator ""
@@ -97,12 +132,12 @@ module Bioinform
         option_parser.parse!(argv)
         @arguments = argv
       end
-      
-      
+
+
       def self.main(argv)
         self.new.main(argv)
       end
-      
+
     end
   end
-end
+end