bio-alignment 0.0.5 → 0.0.6

data/Gemfile

@@ -1,13 +1,14 @@
 source "http://rubygems.org"
 gem "bio-logger"
-gem "bio", ">= 1.4.2"      # for translation tables 
+gem "bio", ">= 1.4.2"      # for translation tables, BioRuby compat and Newick parser
 
 # Add dependencies to develop your gem here.
 # Include everything needed to run rake, tests, features, etc.
 group :development do
-  gem "bio-bigbio", "> 0.1.3"         # for FASTA files in tests
+  gem "rake"
+  gem "bio-bigbio", "> 0.1.3"         # for reading FASTA files in tests
   gem "cucumber", ">= 0"
   gem "rspec", "~> 2.3.0"
-  gem "bundler", "~> 1.0.0"
-  gem "jeweler", "~> 1.7.0"
+  gem "bundler", ">= 1.0.21"
+  gem "jeweler"
 end

data/README.md

@@ -29,6 +29,7 @@ aligmment (note codon gaps are represented by '---')
   require 'bio-alignment'
   require 'bigbio' # Fasta reader and writer
 
+  include Bio::BioAlignment
   aln = Alignment.new
   fasta = FastaReader.new('codon-alignment.fa')
   fasta.each do | rec |
@@ -81,11 +82,13 @@ BioAlignment supports adding BioRuby's Bio::Sequence objects:
 
 ```ruby
   require 'bio'  # BioRuby
+  require 'bio-alignment'
   require 'bio-alignment/bioruby' # make Bio::Sequence enumerable
-  
+  include Bio::BioAlignment
+
   aln = Alignment.new
-  aln << Bio::Sequence::NA.new("atgcatgcaaaa")
-  aln << Bio::Sequence::NA.new("atg---tcaaaa")
+  aln.sequences << Bio::Sequence::NA.new("atgcatgcaaaa")
+  aln.sequences << Bio::Sequence::NA.new("atg---tcaaaa")
 ```
 
 and we can transform BioAlignment into BioRuby's Bio::Alignment and
@@ -146,21 +149,103 @@ version of pal2nal includes validation
 
 resulting in the codon alignment.
 
-### Alignment editing
+### Phylogeny
+
+BioAlignment has support for attaching a phylogentic tree to an
+alignment, and traversing the tree.
+
+### Alignment marking/masking/editing
+
+One of the primary reasons for creating BioAlignment is to provide
+easy ways of editing alignments using a functional style of
+programming. Primitives are provided which take out much of the
+plumbing, such as maintaining row/column/element state, and allow
+copy-on-edit (so no conflicts arise in concurrent code). For example,
+to walk an alignment by row, and update the row state, you can mark
+all rows for deletion which contain many gaps
+
+```ruby
+  include MarkRows
+  mark_rows { |rowstate,row|  # for every row/sequence
+    num = row.count { |e| e.gap? }
+    if (num.to_f/row.length) > 0.5
+      rowstate.delete!  # mark row for deletion
+    end
+    rowstate   # returns the updated row state
+  }
+```
+
+next, return a (deep) copy of the original alignment with the rows
+that are not marked for deletion
+
+```ruby
+  aln2 = aln.rows_where { |row| !row.state.deleted? }
+```
+
+The general idea is that there are many potential ways of selecting
+rows, and changing some state. The 'mark_rows' function/iterator takes
+care of the plumbing. All the programmer needs to do is to set the
+criterion, in this case a gap percentage, and tell the library what
+state has to change. In this example we only access one row, but you
+can also access the other rows. You won't be surprised that marking
+columns looks much the same
 
-BioAlignment supports multiple alignment editing features, which are
+```ruby
+  include MarkColumns
+  mark_columns { |colstate,col|  # for every column
+    num = col.count { |e| e.gap? }
+    if (num.to_f/col.length) > 0.5
+      colstate.delete! 
+    end
+    colstate
+  }
+```
+
+''count'' is one of the universal functions that counts elements in a
+row, column, or alignment.
+
+Next to modifying the state of rows and columns, you can also access
+the state of alignment elements (i.e. codons, amino acids, nucleotide
+acids). For example, here we mask every element that has a masked
+state
+
+```ruby
+  aln = masked_aln.update_each_element { |e| (e.state.masked? ?  Element.new("X"):e)}
+```
+
+and, here we remove every marked element by turning it into a gap
+
+```ruby
+  aln = marked_aln.update_each_element { |e| (e.state.marked? ? Element.new("-"):e)}
+```
+
+''update_each_element'' visits every element in the MSA, and replaces
+the old with the new. 
+
+It is important to note that, instead of directly editing alignments
+in place, this module always makes it a two step process. First items
+are masked/marked through the state of the rows/columns/elements, next
+the alignment is rewritten using this state. The advantage of using an
+intermediate state is that the state can be queried for creating (for
+example) nice output/graphics, using both the original and changed
+alignments. For example, it is really easy to create a nice output
+showing which columns were deleted in the original alignment, or which
+amino acids were masked. Still, methods are available, which hide the
+two step process, as seen in the next example.
+
+BioAlignment supports many alignment editing features, which are
 listed
 [here](https://github.com/pjotrp/bioruby-alignment/tree/master/features/edit).
-Each edition feature is added at runtime(!) Example:
+An edit feature is added at runtime(!) Example:
 
 ```ruby
   require 'bio-alignment/edit/del_bridges'
 
-  aln.extend DelBridges   # bring the module into scope
-  aln2 = aln.clean(50)    # execute the alignment editor
+  aln.extend DelBridges         # mix the module into the object 
+  aln2 = aln.del_bridges        # execute the alignment editor
 ```
 
-
+where aln2 is a copy of aln with bridging columns deleted.
 
 ### See also
 

data/Rakefile

@@ -35,7 +35,8 @@ require 'cucumber/rake/task'
 Cucumber::Rake::Task.new do |features|
 end
 
-task :default => [ :cucumber, :spec ]
+task :test => [ :spec, :cucumber ] 
+task :default => [ :test ]
 
 require 'rdoc/task'
 Rake::RDocTask.new do |rdoc|

data/VERSION

@@ -1 +1 @@
-0.0.5
+0.0.6

data/doc/bio-alignment-design.md

@@ -70,7 +70,7 @@ acid with
     print codons.seq[0].to_aa
 ```
 
-in fact, because Sequence is indexable we can write directly
+in fact, because Sequence is index-able we can write directly
 
 ```ruby
     print codons[0].to_aa        # 'M'
@@ -94,6 +94,16 @@ element or a gap. Also it should respond to the to_s method.
 An element can contain any pay load.  If a list of attributes exists
 in the sequence object, it can be used.
 
+## Elements and CodonSequence
+
+Where the Sequence class is the most basic String representation of a sequence, we
+also have the Elements class, which allows each element in a coding sequence to 
+carry state.
+
+The third list type we normally use in an Alignment, next to Sequence and
+Elements, is the CodonSequence (remember, you can easily roll your own Sequence
+type).
+
 ## Column
 
 The column list tracks the columns of the alignment. The requirement
@@ -135,26 +145,80 @@ The Matrix can be accessed in transposed fashion, but accessing the normal
 matrix and transposed matrix at the same time is not supported.  Matrix is not
 designed to be transaction safe - though you can copy the Matrix any time.
 
+
 ## Adding functionality
 
-To ascertain that the basic BioAlignment does not get polluted, extra functionality
-is added by Modules. These modules can be added at run time(!) One advantage is
-that there is less name space pollution, the other is that different implementations
-can be plugged in - using the same interface. For example, here we are going to
-use an alignment editor named DelBridges, which has a method named clean:
+To ascertain that the basic BioAlignment implementation does not get
+polluted, extra functionality is added by using modules. These
+modules can be added at run time(!) One advantage is that there is
+less name space pollution, the other is that different implementations
+can be plugged in - using the same interface. For example, here we are
+going to use an alignment editor named DelBridges, which has a method
+named del_bridges:
 
 ```ruby
   require 'bio-alignment/edit/del_bridges'
 
   aln = Alignment.new(string.split(/\n/))
   aln.extend DelBridges   # bring the module into scope
-  aln2 = aln.clean
+  aln2 = aln.del_bridges
+```
+
+in other words, the functionality in DelBridges gets attached to the
+aln instance at run time, without affecting any other instantiated
+object(!) Also, when not requiring 'bio-alignment/edit/del_bridges',
+the functionality is never visible, and never added to the
+environment. This type of runtime plugin is something you can only do
+in a dynamic language.
+
+Likewise you may have your own sequence objects in an alignment. To register
+deletion state, simply extend the sequence with the RowState module:
+
+```ruby
+  require 'bio-alignment/state'
+  bioseq = Bio::Sequence::NA.new("AGCT")
+  bioseq.extend(State)          # add state
+  bioseq.state = RowState.new   # set state
+  p mysequence.state.deleted?   # query state
+  > false
 ```
 
-in other words, the functionality in DelBridges gets attached to the aln
-instance at run time, without affecting any other Alignment object(!) Also,
-when not requiring 'bio-alignment/edit/del_bridges', the functionality is never
-visible, and never added to the environment.
+That is impressive - the BioRuby Sequence has no deletion state facility. We
+just added that, and it can even be used in BioAlignment editors which require
+such a state object. See also the scenario "Give deletion state to a
+Bio::Sequence object" in the bioruby.feature.
+
+Note: if we wanted only to allow one plugin per instance at a time, we can
+create a generic interface with a method of the same name for every
+plugged in module. This ascertains that the same method can not be invoked from
+multiple plugins (by default).
+
+## Adding Phylogenetic support
+
+MSAs often come with phylogenetic trees. Not to add this functionality by default,
+we extend BioAlignment with BioAlignment::AlignmentTree when a tree is plugged in
+with the add_tree method.
+
+## Methods returning alignments and concurrency
+
+When an alignment gets changed, e.g. by one of the editing modules, the
+original is copied using the 'clone' method. The idea is never to share data in
+this library. Ruby does not really have guaranteed immutable data, so the only
+safe way to write concurrent code is to copy all data before changing. The
+'clone' methods implemented in the Alignment class are 'deep' clones.
+
+Not only is copying a good idea for concurrency (and lazy caching of
+values), but it also allows one to write succinct and descriptive code
+in functional style, such as
+
+```ruby
+    aln2 = aln.mark_bridges.columns_where { |col| !col.state.deleted? }
+```
 
+where aln2 is a copy (of aln) with columns removed that were marked for
+deletion.  In other words, we apply ''Functional programming in Ruby.'' If
+functions can be easily 'piped', and code can be easily copy and pasted into
+different algorithms, it is likely that the module is written in a functional
+style.
 
 Copyright (C) 2012 Pjotr Prins <pjotr.prins@thebird.nl>

data/features/bioruby-feature.rb

@@ -82,3 +82,20 @@ Then /^I should have a BioRuby Bio::Alignment$/ do
   @bioruby_alignment.consensus_iupac[0..8].should == '???????v?'
 end
 
+Given /^I have a BioRuby sequence object$/ do
+  @bioseq = Bio::Sequence::NA.new("AGCT")
+end
+
+When /^I add RowState$/ do
+  require 'bio-alignment/state'
+  @bioseq.extend State
+  @bioseq.state = RowState.new
+  @bioseq.state.deleted?.should == false
+end
+
+Then /^I should be able to change the delete state$/ do
+  @bioseq.state.delete!
+  @bioseq.state.deleted?.should == true
+end
+
+

data/features/bioruby.feature

@@ -1,9 +1,9 @@
+@bioruby
 Feature: BioAlignment should play with BioRuby 
   In order to use BioRuby functionality
   I want to convert BioAlignment to Bio::Alignment
   And I want to support Bio::Sequence objects
 
-  @bioruby
   Scenario: Use Bio::Sequence to fill BioAlignment
     Given I have multiple Bio::Sequence objects
     When I assign BioAlignment
@@ -22,3 +22,8 @@ Feature: BioAlignment should play with BioRuby
     Given I have a BioAlignment
     When I convert
     Then I should have a BioRuby Bio::Alignment
+
+  Scenario: Give deletion state to a Bio::Sequence object
+    Given I have a BioRuby sequence object
+    When I add RowState
+    Then I should be able to change the delete state

data/features/columns-feature.rb

@@ -7,6 +7,8 @@ When /^I fetch a column$/ do
   column = @aln.columns[3]
   column.should_not be_nil
   column[0].to_s.should == 'cga'
+  # ascertain the columns are the same
+  @aln.columns[3].should == column
 end
 
 When /^I inject column state$/ do

data/features/edit/del_bridges-feature.rb

@@ -7,15 +7,19 @@ end
 
 When /^I apply the bridge rule$/ do
   @aln.extend DelBridges
-  aln2 = @aln.clean
+  @aln2 = @aln.mark_bridges
 end
 
 Then /^it should have removed (\d+) bridges$/ do |arg1, string|
-  pending # express the regexp above with the code you wish you had
+  check_aln = Alignment.new(string.split(/\n/))
+  new_aln = @aln.del_bridges
+  new_aln.to_s.should == check_aln.to_s
 end
 
 Then /^I should be able to track removed columns$/ do
-  pending # express the regexp above with the code you wish you had
+  @aln2.columns.count { |col| col.state.deleted? }.should == 6
+  @aln2.columns[0].state.deleted?.should == true
+  @aln2.columns[8].state.deleted?.should_not == true
 end
 
 

data/features/edit/del_bridges.feature

@@ -5,7 +5,6 @@ Feature: Alignment editing, the bridge rule
 
   The dropped columns are tracked by the table columns.
 
-  @dev
   Scenario: Apply bridge rule to an amino acid alignment
     Given I have a bridged alignment
       """
@@ -20,7 +19,7 @@ Feature: Alignment editing, the bridge rule
       -------------IFHAVR-TC-HP-----------------
       """
     When I apply the bridge rule
-    Then it should have removed 4 bridges
+    Then it should have removed 6 bridges
       """
       SNSFSRPTIIFSGCSTACSGKSELVCGFRSFMLSDV
       SNSFSRPTIIFSGCSTACSGKSEQVCGFR---LSDV

data/features/edit/del_non_informative_sequences-feature.rb

@@ -0,0 +1,26 @@
+require 'bio-alignment/edit/del_non_informative_sequences'
+
+Given /^I have a bridged alignment containing unknown amino acids$/ do |string|
+  @aln = nil
+  @aln2 = nil
+  @aln = Alignment.new(string.split(/\n/))
+  @aln.extend DelNonInformativeSequences
+end
+
+When /^I apply the non\-informative sequence rule$/ do
+  @aln2 = @aln.mark_non_informative_sequences
+end
+
+Then /^it should have removed two rows$/ do |string|
+  check_aln = Alignment.new(string.split(/\n/))
+  new_aln = @aln.del_non_informative_sequences
+  new_aln.to_s.should == check_aln.to_s
+end
+
+Then /^I should be able to track removed non\-informative rows$/ do
+  @aln2.rows.count { |row| row.state.deleted? }.should == 2
+  @aln2.rows[0].state.deleted?.should == false
+  @aln2.rows[3].state.deleted?.should == true
+  @aln2.rows[4].state.deleted?.should == true
+end
+

data/features/edit/del_non_informative_sequences.feature

@@ -2,3 +2,22 @@ Feature: Remove non-informative sequences
 
   After alignment cleaning, it may be we have non-informative sequences. These
   can be removed from the alignment.
+
+  Scenario: Apply non informative sequence rule to an amino acid alignment
+    Given I have a bridged alignment containing unknown amino acids
+      """
+      SSIISNSFSRPTIIFSGCSTACSGK--SEQVCGFR---LSDV
+      SSIISNSFSRPTIIFSGCSTACSGKLTSEQVCGFR---LSDV
+      ----------PTIIFSGCSKACSGK-----VCGIFHAVRSFM
+      ----------XTIXXXXXXXXXSGK--SELXXXXXSFXXXXV
+      -------------IFHAVR-TC-HP-----------------
+      """
+    When I apply the non-informative sequence rule
+    Then it should have removed two rows
+      """
+      SSIISNSFSRPTIIFSGCSTACSGK--SEQVCGFR---LSDV
+      SSIISNSFSRPTIIFSGCSTACSGKLTSEQVCGFR---LSDV
+      ----------PTIIFSGCSKACSGK-----VCGIFHAVRSFM
+      """
+    Then I should be able to track removed non-informative rows
+

data/features/edit/del_short_sequences-feature.rb

@@ -0,0 +1,21 @@
+require 'bio-alignment/edit/del_short_sequences'
+
+When /^I apply the short sequence rule$/ do
+  @aln.extend DelShortSequences
+  @aln2 = @aln.mark_short_sequences
+end
+
+Then /^it should have removed one row$/ do |string|
+  check_aln = Alignment.new(string.split(/\n/))
+  new_aln = @aln.del_short_sequences
+  print new_aln.to_s
+  new_aln.to_s.should == check_aln.to_s
+end
+
+Then /^I should be able to track removed rows$/ do
+  @aln2.rows.count { |row| row.state.deleted? }.should == 1
+  @aln2.rows[0].state.deleted?.should == false
+  @aln2.rows[4].state.deleted?.should == true
+end
+
+