npsearch 0.0.1

data/README.md

@@ -0,0 +1,68 @@
+# NeuroPeptideSearch (NpSearch)
+[![Gem Version](https://badge.fury.io/rb/NpSearch.svg)](http://badge.fury.io/rb/NpSearch)
+[![Build Status](https://travis-ci.org/IsmailM/NeuroPeptideSearch.svg?branch=master)](https://travis-ci.org/IsmailM/NeuroPeptideSearch)
+[![Dependency Status](https://gemnasium.com/IsmailM/NeuroPeptideSearch.svg)](https://gemnasium.com/IsmailM/NeuroPeptideSearch)
+[![Inline docs](http://inch-ci.org/github/IsmailM/NeuroPeptideSearch.png?branch=master)](http://inch-ci.org/github/IsmailM/NeuroPeptideSearch)
+
+> A tool to identify noval Neuropeptides.
+
+NpSearch (NeuroPeptideSearch) is a program that searches for potential neuropeptides precursors based on the motifs commonly found on a neuropeptide. Ideally, the input would be transcriptome or protein data since there are no introns to worry about and the signal peptide would be attached to the front of the precursor.
+
+Currently, the program produces a long list of sequences that fulfil all the requirements to be a potential neuropeptide. This list needs to be further analysed to find potential neuropeptides. Future versions of the program will automatically analyse the output file and extract a list of highly likely neuropeptides.
+
+NpSearch produces a number of files - the final output files is produced as a fasta file and as a colour coded html file that can be opened by any web browser or even in a word processor.
+
+Note: For this program to work, you will need to obtain a copy of Signal P 4.1 from cbs at "http://www.cbs.dtu.dk/cgi-bin/nph-sw_request?signalp" and link this to the program. Alternatively you will require an output text file from the Signal P which you can input into the program.
+
+** Currently only supported on Mac OS & Linux
+
+If you use this program, please cite us:
+
+Moghul I, Rowe M, Priyam A, ELphick M & Wurm Y <em>(in prep)</em> NpSearch: A Tool to Identify Novel Neuropeptides
+
+## Installation
+
+1. Simply open the terminal and type this
+```
+    $ gem install npsearch
+```
+## Usage
+
+    * Usage: npsearch [Options] -i [Input File] -o [Output Folder Name]
+
+    * Mandatory Options:
+
+        -i, --input [file]               The input file (in fasta format). Can be a relative or a full
+                                          path.
+        -o, --output [folder name]       The path to the output folder. This will be created if the
+                                          folder does not exist.
+
+    * Optional Options:
+        -m, --motif [Query Motif]        By default NpSearch only searches for dibasic cleavage site
+                                          ("KR", "RR" or "KK"). This option allows one to change the
+                                          set of cleavage sites to be searched.
+                                          The period "." can be used to denote any character. Multiple
+                                          motifs query can be used by using a pipeline character ("|")
+                                          between each query and putting the motif query in speech marks
+                                          e.g. "KR|RR|R..R"
+                                          Advanced Users: Regular expressions are supported.
+        -c, --cut_off N                  Changes the minimum Open Reading
+                                          Frame from the default 10 amino acid residues to N amino acid
+                                          residues.
+        -s, --signalp_file [file]        Is used to supply the signal peptide results to the program.
+                                          These signal peptide results must be created using the SignalP
+                                          program (Version 4.x), downloadable from CBS. If this argument
+                                          isn't suplied, then NpSearch will try to run a local version
+                                          of the Signal P script.
+        -e, --extract_orf                Only extracts the Open Reading Frames.
+        -v, --verbose                    Provides more information on each step taken in this program.
+        -h, --help                       Display this screen
+            --version                    Shows version
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request

data/Rakefile

@@ -0,0 +1,14 @@
+require 'bundler/gem_tasks'
+require 'rake/testtask'
+
+task default: [:build]
+desc 'Installs the ruby gem'
+task :build do
+  exec("gem build np_search.gemspec && gem install ./NpSearch-#{NpSearch::VERSION}.gem")
+end
+
+task :test do
+  Rake::TestTask.new do |t|
+    t.pattern = 'test/test_np_search.rb'
+  end
+end

data/bin/npsearch

@@ -0,0 +1,165 @@
+#!/usr/bin/env ruby
+require 'optparse'
+
+require 'npsearch'
+require 'npsearch/arg_validator'
+require 'npsearch/version'
+
+opt = {}
+optparse = OptionParser.new do |opts|
+  opts.banner = <<Banner
+
+* Usage: npsearch [Options] -i [Input File] -o [Output Folder Name]
+
+* Mandatory Options:
+
+Banner
+
+  opt[:input_file] = nil
+  opts.on('-i', '--input [file]', 'Path to the input fasta file') do |f|
+    opt[:input_file] = f
+  end
+
+  opts.separator ''
+  opts.separator '* Optional Options:'
+
+  opt[:motif] = 'KR|RR|KK'
+  opts.on('-m', '--motif [Query Motif]', 'By default NpSearch only searches',
+          ' for dibasic cleavage site ("KR", "RR" or "KK"). This option allows',
+          ' one to change the set of cleavage sites to be searched.',
+          ' The period "." can be used to denote any character. Multiple',
+          ' motifs query can be used by using a pipeline character ("|")',
+          ' between each query and putting the motif query in speech marks',
+          ' e.g. "KR|RR|R..R"',
+          ' Advanced Users: Regular expressions are supported.') do |motif|
+    opt[:motif] = motif
+  end
+
+  opt[:cut_off] = 10
+  opts.on('-c', '--cut_off N', Integer, 'Changes the minimum Open Reading',
+          ' Frame from the default 10 amino acid residues to N amino acid',
+          ' residues.') do |n|
+    opt[:cut_off] = n
+  end
+
+  opt[:signalp_file] = nil
+  opts.on('-s', '--signalp_file [file]',
+          'Is used to supply the signal peptide results to the program. These',
+          ' signal peptide results must be created using the SignalP program',
+          " (Version 4.x), downloadable from CBS. If this argument isn't ",
+          ' suplied, then NpSearch will try to run a local version of the',
+          ' Signal P script.') do |signalp_file|
+    opt[:signalp_file] = signalp_file
+  end
+
+  opt[:extract_orf] = false
+  opts.on('-e', '--extract_orf', 'Only extracts the Open Reading Frames.') do
+    opt[:extract_orf] = true
+  end
+
+  opt[:verbose] = false
+  opts.on('-v', '--verbose', 'Provides more information on each step taken',
+          ' in this program.') do
+    opt[:verbose] = true
+  end
+
+  opts.on('-h', '--help', 'Display this screen') do
+    puts opts
+    exit
+  end
+
+  opts.on('--version', 'Shows version') do
+    puts NpSearch::VERSION
+    exit
+  end
+end
+optparse.parse!
+
+NpSearch.init(opt)
+NpSearch.run
+
+
+
+
+# ############# Argument Validation...##############
+# arg_vldr = NpSearch::ArgValidators.new(opt[:verbose])
+# input_type = arg_vldr.arg(opt[:motif], opt[:input], opt[:output_dir],
+#                           opt[:cut_off], opt[:extract_orf], opt[:signalp_file],
+#                           optparse.help)
+
+# ############# General Validation...##############
+# vldr = NpSearch::Validators.new
+# vldr.output_dir(opt[:output_dir])
+# if opt[:signalp_file].nil? && opt[:extract_orf] == false
+#   sp_dir = vldr.signalp_dir
+# end
+
+# ############# Converting input file to Bio::FastaFormat. #############
+# input_read = NpSearch::Input.read(opt[:input], input_type)
+
+# ############# Extract_ORF #############
+# if input_type == 'genetic'
+#   # Translate Sequences in all 6 frames
+#   translated = NpSearch::Translation.translate(input_read)
+#   translated.to_fasta('translated seq.', "#{opt[:output_dir]}/1_protein.fa")
+#   # Extract all possible ORF that are longer than the ORF_min_length
+#   orf = NpSearch::Translation.extract_orf(translated, opt[:cut_off])
+#   orf.to_fasta('Open Reading Frames', "#{opt[:output_dir]}/2_orf.fa")
+
+#   if opt[:extract_orf]
+#     puts "\nSuccess: All output files created in the directory:" \
+#          "#{opt[:output_dir]}'.\n "
+#     exit
+#   end
+# end
+
+# ############# Setting up more variables...##############
+# if opt[:motif] == 'neuro_clv'
+#   motif = 'KK|KR|RR|' \
+#           'R..R|R....R|R......R|H..R|H....R|H......R|K..R|K....R|K......R'
+# else
+#   motif = opt[:motif]
+# end
+# vldr.motif_type(motif)
+
+# if input_type == 'genetic'
+#   sp_input_file = "#{opt[:output_dir]}/2_orf.fa"
+#   sp_hash = orf
+#   file_number = 3
+# else # i.e. if the input is protein
+#   sp_input_file = opt[:input]
+#   sp_hash = input_read
+#   file_number = 1
+# end
+
+# if opt[:signalp_file].nil?
+#   sp_out_file = "#{opt[:output_dir]}/#{file_number}_signalp_out.txt"
+#   file_number += 1
+#   NpSearch::Signalp.signalp(sp_dir, sp_input_file, sp_out_file)
+# else
+#   sp_out_file = opt[:signalp_file]
+#   file_number = 1
+# end
+
+# ############# Signal P Results file Validation #############
+# vldr.sp_results(sp_out_file)
+
+# ############# Extract sequences with a signal peptide #############
+# secretome = NpSearch::Analysis.parse(sp_out_file, sp_hash, motif)
+# secretome.to_fasta('secretome file',
+#                    "#{opt[:output_dir]}/#{file_number}_secretome.fa")
+# file_number += 1
+
+# ############# Remove any duplicate data #############
+# flattened_seq = NpSearch::Analysis.flattener(secretome)
+
+# ############# Creating Output Files #############
+# flattened_seq.to_fasta('fasta output file',
+#                        "#{opt[:output_dir]}/#{file_number}_output.fa")
+# flattened_seq.to_html(motif,
+#                       "#{opt[:output_dir]}/#{file_number}_output.html")
+
+# ############# Success #############
+# puts # a blank line.
+# puts "Success: All output files created in the directory:'#{opt[:output_dir]}'."
+# puts # a blank line

data/lib/npsearch.rb

@@ -0,0 +1,96 @@
+require 'bio'
+require 'fileutils'
+
+# require 'npsearch/arg_validator'
+require 'npsearch/logger'
+require 'npsearch/sequence'
+require 'npsearch/signalp'
+require 'npsearch/pool'
+
+# Top level module / namespace.
+module NpSearch
+  class <<self
+    MIN_ORF_SIZE = 40 # amino acids (including potential signal peptide)
+
+    attr_accessor :opt
+    attr_accessor :sequences
+
+    def logger
+      @logger ||= Logger.new(STDERR, @opt[:verbose])
+    end
+
+    def init(opt)
+      # @opt = args_validation(opt)
+      @opt        = opt
+      @sequences  = []
+      @opt[:num_threads] = 8
+      @opt[:type] = guess_sequence_type
+      @opt[:signalp_path] = '/Volumes/Data/programs/signalp-4.1/signalp'
+      @pool       = Pool.new(@opt[:num_threads]) if @opt[:num_threads] > 1
+    end
+
+    def run
+      iterate_input_file
+      # score_sequence
+      # scan(?<=(KR|RR|KK))(\w+?)(?=(KR|RR|KK|$))
+      @sequences.each { |s| puts ">#{s.id}\n#{s.seq}" }
+    end
+
+    private
+
+    def iterate_input_file
+      biofastafile = Bio::FlatFile.open(Bio::FastaFormat, @opt[:input_file])
+      biofastafile.each_entry do |entry|
+        if @opt[:num_threads] > 1
+          @pool.schedule(entry) { |e| initialise_seqs(e) }
+        else
+          initialise_seqs(entry)
+        end
+      end
+      @pool.shutdown if @opt[:num_threads] > 1
+    end
+
+    def initialise_seqs(entry)
+      if @opt[:type] == :protein
+        initialise_protein_seq(entry.entry_id, entry.aaseq)
+      else
+        initialise_transcriptomic_seq(entry.entry_id, entry.naseq)
+      end
+    end
+
+    def initialise_protein_seq(id, seq)
+      sp = Signalp.analyse_sequence(seq)
+      @sequences << Sequence.new(id, seq, sp) if sp[:sp] == 'Y'
+    end
+
+    def initialise_transcriptomic_seq(id, naseq)
+      (1..6).each do |f|
+        translated_seq = naseq.translate(f)
+        orfs = translated_seq.to_s.scan(/(?=(M\w{#{MIN_ORF_SIZE},}))./).flatten
+        initialise_orfs(id, orfs, f)
+      end
+    end
+
+    def initialise_orfs(id, orfs, frame)
+      idx = 0
+      orfs.each do |orf|
+        sp = Signalp.analyse_sequence(orf)
+        next if sp[:sp] == 'N'
+        seq                  = Sequence.new(id, orf, sp)
+        seq.translated_frame = frame
+        seq.orf_index        = idx
+        @sequences << seq
+        idx += 1
+      end
+    end
+
+    def guess_sequence_type
+      fasta_content = IO.binread(@opt[:input_file])
+      # removing non-letter and ambiguous characters
+      cleaned_sequence = fasta_content.gsub(/[^A-Z]|[NX]/i, '')
+      return nil if cleaned_sequence.length < 10 # conservative
+      type = Bio::Sequence.new(cleaned_sequence).guess(0.9)
+      (type == Bio::Sequence::NA) ? :nucleotide : :protein
+    end
+  end
+end

data/lib/npsearch/arg_validator.rb

@@ -0,0 +1,264 @@
+module NpSearch
+  class ArgValidators
+    
+    
+    # Changes the logger level to output extra info when the verbose option is
+    #   true.
+    def initialize(verbose_opt)
+      LOG.level = Logger::INFO if verbose_opt == true
+    end
+
+    # Runs all the arguments method...
+    def arg(motif, input, output_dir, orf_min_length, extract_orf,
+            signalp_file, help_banner)
+      comp_arg(input, motif, output_dir, extract_orf, help_banner)
+      input_type = guess_input_type(input)
+      extract_orf_conflict(input_type, extract_orf)
+      input_sp_file_conflict(input_type, signalp_file)
+      orf_min_length(orf_min_length)
+      input_type
+    end
+
+    # Ensures that the compulsory input arguments are supplied...
+    def comp_arg(input, motif, output_dir, extract_orf, help_banner)
+      comp_arg_error(motif, 'Query Motif ("-m" option)') if extract_orf == false
+      comp_arg_error(input, 'Input file ("-i option")')
+      comp_arg_error(output_dir, 'Output Folder ("-o" option)')
+      return unless input.nil? || (motif.nil? && extract_orf == false)
+      puts help_banner
+      exit
+    end
+
+    # Ensures that a message is provided for all missing compulsory args.
+    #   Run from comp_arg method
+    def comp_arg_error(arg, message)
+      puts 'Usage Error: No ' + message + ' is supplied' if arg.nil?
+    end
+
+    # Guesses the type of data within the input file on the first 100 lines of
+    #   the file (ignores all identifiers (lines that start with a '>').
+    #   It has a 80% threshold.
+    def guess_input_type(input_file)
+      input_file_format(input_file)
+      sequences = []
+      File.open(input_file, 'r') do |file_stream|
+        file_stream.readlines[0..100].each do |line|
+          sequences << line.to_s unless line.match(/^>/)
+        end
+      end
+      type = Bio::Sequence.new(sequences).guess(0.8)
+      if type == Bio::Sequence::NA
+        input_type = 'genetic'
+      elsif type == Bio::Sequence::AA
+        input_type = 'protein'
+      end
+      input_type
+    end
+
+    # Ensures that the input file a) exists b) is not empty and c) is a fasta
+    #   file. Run from the guess_input_type method.
+    def input_file_format(input_file)
+      unless File.exist?(input_file)
+        fail ArgumentError("Critical Error: The input file '#{input_file}'" \
+                           ' does not exist.')
+      end
+      if File.zero?(input_file)
+        fail ArgumentError("Critical Error: The input file '#{input_file}'" \
+                            ' is empty.')
+      end
+      unless File.probably_fasta?(input_file)
+        fail ArgumentError("Critical Error: The input file '#{input_file}'" \
+                            ' does not seem to be in fasta format. Only' \
+                            ' input files in fasta format are supported.')
+      end
+    end
+
+    # Ensures that the extract_orf option is only used with genetic data.
+    def extract_orf_conflict(input_type, extract_orf)
+      return unless input_type == 'protein' && extract_orf == true
+      fail ArgumentError('Usage Error: Conflicting arguments detected:' \
+                          ' Protein data detected within the input file,' \
+                          ' when using the  Extract_ORF option (option' \
+                          ' "-e"). This option is only available when' \
+                          ' input file contains genetic data.')
+    end
+
+    # Ensures that the protein data (or open reading frames) are supplied as
+    #   the input file when the signal p output file is passed.
+    def input_sp_file_conflict(input_type, signalp_file)
+      return unless input_type == 'genetic' && !signalp_file.nil?
+      fail ArgumentError('Usage Error: Conflicting arguments detected' \
+                          ': Genetic data detected within the input file' \
+                          ' when using the Signal P Input Option (Option' \
+                          ' "-s"). The Signal P input Option requires the' \
+                          ' input of two files: the Signal P Script Result' \
+                          ' files (at the "-s" option) and the protein' \
+                          ' data file used to run the Signal P Script.')
+    end
+
+    # Ensures that the ORF minimum length is a number. Any digits after the
+    #   decimal place are ignored.
+    def orf_min_length(orf_min_length)
+      return unless orf_min_length.to_i < 1
+      fail ArgumentError('Usage Error: The Open Reading Frames minimum' \
+                          ' length can only be a full integer.')
+    end
+  end
+
+  class Validators
+    # Checks for the presence of the output directory; if not found, it asks
+    #   the user whether they want to create the output directory.
+    def output_dir(output_dir)
+      unless File.directory? output_dir # If output_dir doesn't exist
+        fail IOError, "\n\nThe output directory deoes not exist\n\n"
+      end
+    rescue IOError
+      puts # a blank line
+      puts 'The output directory does not exist.'
+      puts # a blank line
+      puts "The directory '#{output_dir}' will be created in this location."
+      puts 'Do you to continue? [y/n]'
+      print '> '
+      inp = $stdin.gets.chomp
+      until inp.downcase == 'n' || inp.downcase == 'y' || inp == ''
+        puts # a blank line
+        puts "The input: '#{inp}' is not recognised - 'y' or 'n' are the" \
+             ' only recognisable inputs.'
+        puts 'Please try again.'
+        puts "The directory '#{output_dir}' will be created in this" \
+             ' location.'
+        puts 'Do you to continue? [y/n]'
+        print '> '
+        inp = $stdin.gets.chomp
+      end
+      if inp.downcase == 'y' || inp == ''
+        FileUtils.mkdir_p "#{output_dir}"
+        puts 'Created output directory...'
+      elsif inp.downcase == 'n'
+        raise ArgumentError('Critical Error: An output directory is' \
+                            ' required; please create an output directory' \
+                            ' and then try again.')
+      end
+    end
+
+    # Ensures that the Signal P Script is present. If not found in the home
+    #   directory, it asks the user for its location.
+    def signalp_dir
+      signalp_dir = "#{Dir.home}/SignalPeptide"
+      if File.exist? "#{signalp_dir}/signalp"
+        signalp_directory = signalp_dir
+      else
+        begin
+          fail IOError('The Signal P Script directory cannot be found at' \
+                        " the following location: '#{signalp_dir}/'.")
+        rescue IOError
+          puts # a blank line
+          puts 'Error: The Signal P Script directory cannot be found at the' \
+               " following location: '#{signalp_dir}/'."
+          puts # a blank line
+          puts 'Please enter the full path or a relative path to the Signal' \
+               ' P Script directory (i.e. to the folder containing the' \
+               ' Signal P script). Refer to the online tutorial for more help'
+          print '> '
+          inp = $stdin.gets.chomp
+          until (File.exist? "#{signalp_dir}/signalp") ||
+                (File.exist? "#{inp}/signalp")
+            puts # a blank line
+            puts 'The Signal P directory cannot be found at the following' \
+                 " location: '#{inp}'"
+            puts 'Please enter the full path or a relative path to the Signal' \
+                 ' Peptide directory again.'
+            print '> '
+            inp = $stdin.gets.chomp
+          end
+          signalp_directory = inp
+          puts # a blank line
+          puts "The Signal P directory has been found at '#{signalp_directory}'"
+          FileUtils.ln_s "#{signalp_directory}", "#{Dir.home}/SignalPeptide",
+                         force: true
+          puts # a blank line
+        end
+      end
+      signalp_directory
+    end
+
+    # Ensures that the supported version of the Signal P Script has been linked
+    #   to NpSearch. Run from the 'sp_results' method.
+    def sp_version(input_file)
+      File.open(input_file, 'r') do |file_stream|
+        first_line = file_stream.readline
+        if first_line.match(/# SignalP-4.1/)
+          return true
+        else
+          return false
+        end
+      end
+    end
+
+    # Ensures that the critical columns in the tabular results produced by the
+    #   Signal P script are conserved. Run from the 'sp_results' method.
+    def sp_column(_input_file)
+      File.open('signalp_out.txt', 'r') do |file_stream|
+        secondline = file_stream.readlines[1]
+        row = secondline.gsub(/\s+/m, ' ').chomp.split(' ')
+        if row[1] != 'name' && row[4] != 'Ymax' && row[5] != 'pos' &&
+           row[9] != 'D'
+          return true
+        else
+          return false
+        end
+      end
+    end
+
+    # Ensure that the right version of the Signal P script is used (via
+    #   'sp_version' Method). If the wrong signal p script has been linked to
+    #   NpSearch, check whether the critical columns in the tabular results
+    #   produced by the Signal P Script are conserved (via 'sp_column'
+    #   Method).
+    def sp_results(signalp_output_file)
+      return if sp_version(signalp_output_file)
+      # i.e. if Signal P is the wrong version
+      if sp_column(signalp_output_file) # If wrong version but correct columns
+        puts # a blank line
+        puts 'Warning: The wrong version of signalp has been linked.' \
+             ' However, the signal peptide output file still seems to' \
+             ' be in the right format.'
+      else
+        puts # a blank line
+        puts 'Warning: The wrong version of the signal p has been linked' \
+             ' and the signal peptide output is in an unrecognised format.'
+        puts 'Continuing may give you meaningless results.'
+      end
+      puts # a blank line
+      puts 'Do you still want to continue? [y/n]'
+      print '> '
+      inp = $stdin.gets.chomp
+      until inp.downcase == 'n' || inp.downcase == 'y'
+        puts # a blank line
+        puts "The input: '#{inp}' is not recognised - 'y' or 'n' are the" \
+             ' only recognisable inputs.'
+        puts 'Please try again.'
+      end
+      if inp.downcase == 'y'
+        puts 'Continuing.'
+      elsif inp.downcase == 'n'
+        fail IOError('Critical Error: NpSearch only supports SignalP 4.1' \
+                      ' (downloadable form CBS) Please ensure the version' \
+                      ' of the signal p script is downloaded.')
+      end
+    end
+
+    # Guesses the type of the data in the supplied motif. It ignores all
+    #   non-word characters (e.g. '|' that is used for regex). It has a 90%
+    #   threshold.
+    def motif_type(motif)
+      motif_seq = Bio::Sequence.new(motif.gsub(/\W/, ''))
+      type = motif_seq.guess(0.9)
+      return unless type.to_s != 'Bio::Sequence::AA'
+      fail IOError('Critical Error: There seems to be an error in' \
+                    ' processing the motif. Please ensure that the motif' \
+                    ' contains amino acid residues that you wish to search' \
+                    ' for.')
+    end
+  end
+end