seqtrimnext 2.0.35 → 2.0.36

data/History.txt

@@ -1,3 +1,7 @@
+=== 2.0.36 2011-06-24
+
+Added clustered installation documentation
+
 === 2.0.35 2011-06-22
 
 FarAdapters improvement

data/README.rdoc

@@ -46,7 +46,15 @@ Once installed, SeqtrimNEXT is very easy to use:
   
 To install core databases (it should be done at installation time):
 
-  $> seqtrimnext -i
+  $> seqtrimnext -i core
+  
+Databases will be installed nearby SeqtrimNEXT by default, but you can override this location by setting the environment variable +BASTDB+. Eg.:
+
+If you with your database installed at /var:
+
+  $> export BLASTDB=/var/DB/formatted
+
+Be sure that this environment variable is always loaded before SeqtrimNEXT execution (Eg.: add it to /etc/profile.local).
 
 To perform an analisys using a predefined template with a FASTQ file format using 4 cpus:
 
@@ -60,7 +68,52 @@ To perform an analisys using a predefined template with a FASTQ file format:
 To get additional help and list available templates and databases:
 
   $> seqtrimnext -h
+
+=== CLUSTERED EXECUTION:
+
+To take full advantage of a clustered installation, you can launch SeqtrimNEXT in distributed mode. You only need to provide it a list of machine names (or IPs) where workers will be launched. 
+
+Setup a workers file like this:
+    
+    machine1
+    machine1
+    machine2
+    machine2
+    machine2
+
+And launch SeqtrimNEXT this way:
+
+    $> seqtrimnext -t genomics_454.txt -Q input_file_in_FASTQ -w workers_file -s 10.0.0
+    
+This will launch 2 workers on machine1 and 3 workers on machine2 using the network whose ip starts with 10.0.0 to communicate.
+
+  
+== TEMPLATE MODIFICATIONS
+
+You can modify any template to fit your workflow. To do this, you only need to copy one of the templates and edit it with a text editor, or simply modify a used_params.txt file that was produced by a previous SeqtrimNEXT execution.
   
+Eg.: If you want to disable repetition removal, do this:
+
+1-Copy the template file you wish to customize and name it params.txt.
+2-Edit params.txt with a text editor
+3-Find a line like this:
+
+remove_clonality = true
+
+
+4-Replace this line with:
+
+remove_clonality = false
+
+5- Launch SeqtrimNEXT with params.txt file instead of a default template:
+
+  $> seqtrimnext -t params.txt -f input_file_in_FASTA -q input_file_in_QUAL
+
+
+
+The same way you can modify any of the parameters. You can find all parameters and their description in any used_params.txt file generated by a previous SeqtrimNEXT execution. Parameters not especified in a template are automatically set to their default value at execution time.
+
+<b>NOTE</b>: The only mandatory parameter is the plugin_list one.
 
 == REQUIREMENTS:
 
@@ -121,6 +174,8 @@ This will install seqtrimnext and all the required gems.
 SeqtrimNEXT needs some core databases to work. To install them:
 
   seqtrimnext -i core
+  
+You can change default database location by setting the environment variable +BASTDB+. Refer to SYNOPSIS for an example.
 
 === Database modifications
 
@@ -139,6 +194,89 @@ Once the databases has been modified, you will need to reformat them by issuing
 Modified databases will be rebuilt.
 
 
+== CLUSTERED INSTALLATION
+
+To install SeqtrimNEXT into a cluster, you need to have the software available on all machines. By installing it on a shared location, or installing it on each cluster node. Once installed, you need to create a init_file where your environment is correctly setup (paths, BLASTDB, etc):
+
+  export PATH=/apps/blast+/bin:/apps/cd-hit/bin
+  export BLASTDB=/var/DB/formatted
+  export SEQTRIMNEXT_INIT=path_to_init_file
+  
+
+And initialize the SEQTRIMNEXT_INIT environment variable on your main node (from where SeqtrimNEXT will be initially launched):
+
+  export SEQTRIMNEXT_INIT=path_to_init_file
+
+If you use any queue system like PBS Pro or Moab/Slurm, be sure to initialize the variables on each submission script. 
+
+<b>NOTE</b>: all nodes on the cluster should use ssh keys to allow SeqtrimNEXT to launch workers without asking for a password.
+
+== SAMPLE INIT FILES FOR CLUSTERED INSTALLATION:
+
+=== Init file
+
+  $> cat stn_init_env 
+
+  source ~latex/init_env
+  source ~ruby19/init_env
+  source ~blast_plus/init_env
+  source ~gnuplot/init_env
+  source ~cdhit/init_env
+
+  export BLASTDB=~seqtrimnext/DB/formatted/
+  export SEQTRIMNEXT_INIT=~seqtrimnext/stn_init_env
+
+
+=== PBS Submission script
+
+  $> cat sample_work.sh 
+  
+  # 40 distributed workers and 1 GB memory per worker:
+  #PBS -l select=40:ncpus=1:mpiprocs=1:mem=1gb
+  # request 10 hours of walltime:
+  #PBS -l walltime=10:00:00
+  # cd to working directory (from where job was submitted)
+  cd $PBS_O_WORKDIR
+
+  # create workers file with assigned node names
+
+  cat ${PBS_NODEFILE} > workers
+
+  # init seqtrimnext
+  source ~seqtrimnext/init_env
+
+  time seqtrimnext -t paired_ends.txt -Q fastq -w workers -s 10.0.0
+
+
+Once this submission script is created, you only need to launch it with:
+
+  qsub sample_work.sh
+
+=== MOAB/SLURM submission script
+
+  $> cat sample_work_moab.sh
+
+  #!/bin/bash 
+  # @ job_name = STN
+  # @ initialdir = .
+  # @ output = STN_%j.out
+  # @ error = STN_%j.err
+  # @ total_tasks = 40
+  # @ wall_clock_limit = 10:00:00
+
+  # guardar lista de workers
+  sl_get_machine_list > workers
+
+  # init seqtrimnext
+  source ~seqtrimnext/init_env
+
+  time seqtrimnext -t paired_ends.txt -Q fastq -w workers -s 10.0.0
+
+Then you only need to submit your job with mnsubmit
+
+  mnsubmit sample_work_moab.sh
+
+
 == LICENSE:
 
 (The MIT License)

data/lib/seqtrimnext/classes/em_classes/seqtrim_worker.rb

@@ -36,7 +36,7 @@ ENV['BLASTDB']=$FORMATTED_DB_PATH
 
 OUTPUT_PATH='output_files'
 
-
+puts "FORMATTED_DB_BLAST in workers: #{$FORMATTED_DB_PATH}"
 # $: << File.expand_path('~/progs/ruby/gems/scbi_mapreduce/lib')
 
 require 'scbi_mapreduce'

data/lib/seqtrimnext.rb

@@ -30,7 +30,7 @@ module Seqtrimnext
   # SEQTRIM_VERSION_STAGE = 'b'
   # SEQTRIM_VERSION = "2.0.0#{SEQTRIM_VERSION_STAGE}#{SEQTRIM_VERSION_REVISION}"
   
-   VERSION = '2.0.35'
+   VERSION = '2.0.36'
   
   SEQTRIM_VERSION = VERSION
   

metadata

@@ -2,7 +2,7 @@
 name: seqtrimnext
 version: !ruby/object:Gem::Version 
   prerelease: 
-  version: 2.0.35
+  version: 2.0.36
 platform: ruby
 authors: 
 - Dario Guerrero & Almudena Bocinos
@@ -10,7 +10,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2011-06-22 00:00:00 Z
+date: 2011-06-24 00:00:00 Z
 dependencies: 
 - !ruby/object:Gem::Dependency 
   name: narray