neptune 0.1.3 → 0.1.4

data/README

@@ -6,13 +6,13 @@ Neptune provides programmers with a simple interface
 by which they can deploy MPI, X10, MapReduce, UPC, and Erlang jobs
 to without needing to know the particulars of the underlying
 cloud platform. You only need to give Neptune your code,
-tell it how many machines to run on and where to put the output:
+tell it how many machines to run on, and where to put the output:
 Neptune handles everything else. No more writing configuration
 files, having to start up NFS on all your machines, yada yada yada.
 Neptune works together with supported cloud platforms (currently
 AppScale is recommended) and can deploy over anything AppScale
 can - Xen or KVM virtual machines as well as Eucalyptus and
-Amazon EC2. There's nothing virtualization specific in there,
+Amazon EC2. There's nothing virtualization-specific in there,
 so in theory any machine installed with the AppScale software
 should work fine.
 
@@ -21,30 +21,33 @@ it also can be used to deploy other types of software. For example,
 Neptune has support for user-specified scaling of the underlying
 cloud platform: users can write Neptune jobs that manually add
 load balancers, application servers, or database servers to a running
-AppScale deployment. Additionally, as of version 0.0.2, remote
-compiling can also be performed: just give Neptune the path to
-the directory you want to compile and be sure to include a Makefile
-in it! Neptune will run 'make' on it (you can specify which target
-to make as well) and return to you a folder containing the standard
-out and standard error of the make command.
+AppScale deployment. Remote compiling can also be performed: just give
+Neptune the path to the directory you want to compile and be sure to
+include a Makefile in it! Neptune will run 'make' on it (you can specify
+which target to make as well) and return to you a folder containing the
+standard out and standard error of the make command.
 
 By default, Neptune jobs store their outputs in the underlying database
-that AppScale is running over. As of Neptune 0.0.5, job outputs can
-also be stored in Amazon S3, Eucalyptus Walrus, and Google Storage.
+that AppScale is running over. Job outputs can also be stored in anything
+that utilizes the Amazon S3 API. We have tested and confirmed compatibility
+with Amazon S3, Eucalyptus Walrus, and Google Storage.
 
-Sample Neptune job scripts can be found in samples. Test cases can
-be found in the test folder, with the standard naming convention
+Sample Neptune job scripts can be found in samples. Integration tests
+can be found in test/integration, with the standard naming convention
 - ts_neptune is the test suite runner, with tc_* containing test
 cases for each type of job that Neptune offers. Before running
 ts_neptune, you should export the environment variable APPSCALE_HEAD_NODE,
 which should be set to the IP address of the AppScale machine that runs
-the Shadow daemon (a.k.a. the Master AppController). Running
-generate_coverage.sh in the top-level directory will run rcov
-and generate the coverage reports automatically via unit tests.
+the Shadow daemon (a.k.a. the Master AppController).
 
-Developed by Chris Bunch as part of the AppScale project.
-See LICENSE for the specifics of the New BSD License
-by which Neptune is released.
+Running generate_coverage.sh in the top-level directory will run rcov
+and generate code coverage reports automatically via unit tests, found in
+test/unit. Running test/unit/ts_all.rb will execute all unit tests, and
+does not require AppScale to be running, or any special environment
+variables to be set.
+
+Developed by Chris Bunch as part of the AppScale project. See LICENSE for
+the specifics of the New BSD License by which Neptune is released.
 
 Check us out on the web:
 
@@ -56,16 +59,18 @@ http://appscale.cs.ucsb.edu
 Contributors welcome! We would love to add support for other
 cloud platforms and test out Neptune more on non-virtualized
 deployments, as well as adding capabilities for other types
-of computation. We would also like to refactor Neptune to use
-symbols instead of instance variables for running jobs: this
-will likely appear in a future release as well.
+of computation. 
 
 Our academic paper on Neptune won best paper at ACM ScienceCloud 2011!
-<a href='http://www.neptune-lang.org/2011/6/Neptune-Picks-up-Best-Paper-at-ScienceCloud-2011'>
-Here's a link</a> to the abstract of the paper and the PDF.
+View the abstract of the paper and the PDF here:
+http://www.neptune-lang.org/2011/6/Neptune-Picks-up-Best-Paper-at-ScienceCloud-2011
 
 Version History:
 
+December 6, 2011 - 0.1.4 released, allowing MPI jobs (and by extension,
+UPC/X10/KDT jobs) to specify command-line arguments via :argv that should be
+passed to the executable.
+
 December 4, 2011 - 0.1.3 released, adding support for KDT (Knowledge Discovery
 Toolkit) and Cicero, a framework for automatic task execution over Google
 App Engine and AppScale.

data/doc/Object.html

@@ -233,7 +233,7 @@ indicates whether or not the compilation was successful.</p>
           <div class="method-source-code"
             id="compile_code-source">
 <pre>
-<span class="ruby-comment"># File lib/neptune.rb, line 290</span>
+<span class="ruby-comment"># File lib/neptune.rb, line 301</span>
 def compile_code(job_data, ssh_args, shadow_ip, shell=<span class="ruby-constant">Kernel</span>.method(:`))
   compiled_location = controller.compile_code(job_data)
 
@@ -333,7 +333,7 @@ job succeeded and if it failed, the reason for it.</p>
           <div class="method-source-code"
             id="get_input-source">
 <pre>
-<span class="ruby-comment"># File lib/neptune.rb, line 236</span>
+<span class="ruby-comment"># File lib/neptune.rb, line 247</span>
 def get_input(job_data, ssh_args, shadow_ip, controller, file=<span class="ruby-constant">File</span>,
   shell=<span class="ruby-constant">Kernel</span>.method(:`))
   result = {:result =&gt; :success}
@@ -394,7 +394,7 @@ end</pre>
           <div class="method-source-code"
             id="get_job_data-source">
 <pre>
-<span class="ruby-comment"># File lib/neptune.rb, line 151</span>
+<span class="ruby-comment"># File lib/neptune.rb, line 162</span>
 def get_job_data(params)
   job_data = {}
   params.each { |k, v|
@@ -470,7 +470,7 @@ not the job completed successfully (success = no errors).</p>
           <div class="method-source-code"
             id="get_std_out_and_err-source">
 <pre>
-<span class="ruby-comment"># File lib/neptune.rb, line 319</span>
+<span class="ruby-comment"># File lib/neptune.rb, line 330</span>
 def get_std_out_and_err(location)
   result = {}
 
@@ -523,7 +523,7 @@ vice-versa).</p>
           <div class="method-source-code"
             id="neptune-source">
 <pre>
-<span class="ruby-comment"># File lib/neptune.rb, line 408</span>
+<span class="ruby-comment"># File lib/neptune.rb, line 419</span>
 def neptune(params)
   puts <span class="ruby-string">&quot;Received a request to run a job.&quot;</span>
   puts params[:type]
@@ -694,6 +694,17 @@ def preprocess_mpi(job_data)
     end
   end
 
+  if job_data[<span class="ruby-string">&quot;@argv&quot;</span>]
+    argv = job_data[<span class="ruby-string">&quot;@argv&quot;</span>]
+    if argv.class != <span class="ruby-constant">String</span> and argv.class != <span class="ruby-constant">Array</span>
+      abort(<span class="ruby-string">&quot;The value specified for :argv must be either a String or Array&quot;</span>) 
+    end
+
+    if argv.class == <span class="ruby-constant">Array</span>
+      job_data[<span class="ruby-string">&quot;@argv&quot;</span>] = argv.join(<span class="ruby-string">' '</span>)
+    end
+  end
+
   return job_data
 end</pre>
           </div>
@@ -729,7 +740,7 @@ specify, convert it to be :trajectories.</p>
           <div class="method-source-code"
             id="preprocess_ssa-source">
 <pre>
-<span class="ruby-comment"># File lib/neptune.rb, line 133</span>
+<span class="ruby-comment"># File lib/neptune.rb, line 144</span>
 def preprocess_ssa(job_data)
   if job_data[<span class="ruby-string">&quot;@simulations&quot;</span>] and job_data[<span class="ruby-string">&quot;@trajectories&quot;</span>]
     abort(<span class="ruby-string">&quot;Both :simulations and :trajectories cannot be specified - use one&quot;</span> +
@@ -778,7 +789,7 @@ as well as information about the node to send the request to.</p>
           <div class="method-source-code"
             id="run_job-source">
 <pre>
-<span class="ruby-comment"># File lib/neptune.rb, line 365</span>
+<span class="ruby-comment"># File lib/neptune.rb, line 376</span>
 def run_job(job_data, ssh_args, shadow_ip, secret,
   controller=<span class="ruby-constant">AppControllerClient</span>, file=<span class="ruby-constant">File</span>)
   controller = controller.new(shadow_ip, secret)
@@ -843,7 +854,7 @@ end</pre>
           <div class="method-source-code"
             id="upload_app_for_cicero-source">
 <pre>
-<span class="ruby-comment"># File lib/neptune.rb, line 337</span>
+<span class="ruby-comment"># File lib/neptune.rb, line 348</span>
 def upload_app_for_cicero(job_data)
   if !job_data[<span class="ruby-string">&quot;@app&quot;</span>]
     puts <span class="ruby-string">&quot;No app specified, not uploading...&quot;</span> 
@@ -899,7 +910,7 @@ end</pre>
           <div class="method-source-code"
             id="validate_storage_params-source">
 <pre>
-<span class="ruby-comment"># File lib/neptune.rb, line 195</span>
+<span class="ruby-comment"># File lib/neptune.rb, line 206</span>
 def validate_storage_params(job_data)
   if !job_data[<span class="ruby-string">&quot;@storage&quot;</span>]
     job_data[<span class="ruby-string">&quot;@storage&quot;</span>] = <span class="ruby-string">&quot;appdb&quot;</span>
@@ -969,7 +980,7 @@ location.</p>
           <div class="method-source-code"
             id="wait_for_compilation_to_finish-source">
 <pre>
-<span class="ruby-comment"># File lib/neptune.rb, line 269</span>
+<span class="ruby-comment"># File lib/neptune.rb, line 280</span>
 def wait_for_compilation_to_finish(ssh_args, shadow_ip, compiled_location,
   shell=<span class="ruby-constant">Kernel</span>.method(:`))
   loop {

data/doc/created.rid

@@ -1,5 +1,5 @@
-Sun, 04 Dec 2011 11:46:34 -0800
+Tue, 06 Dec 2011 14:24:38 -0800
 bin/neptune	Sun, 15 May 2011 22:52:56 -0700
-lib/neptune.rb	Sat, 03 Dec 2011 23:03:43 -0800
+lib/neptune.rb	Tue, 06 Dec 2011 13:21:57 -0800
 lib/app_controller_client.rb	Tue, 25 Oct 2011 23:47:47 -0700
 lib/common_functions.rb	Wed, 26 Oct 2011 10:37:34 -0700

data/doc/lib/neptune_rb.html

@@ -24,7 +24,7 @@
   <div id="metadata">
     <dl>
       <dt class="modified-date">Last Modified</dt>
-      <dd class="modified-date">Sat Dec 03 23:03:43 -0800 2011</dd>
+      <dd class="modified-date">Tue Dec 06 13:21:57 -0800 2011</dd>
 
       
       <dt class="requires">Requires</dt>

data/lib/neptune.rb

@@ -123,6 +123,17 @@ def preprocess_mpi(job_data)
     end
   end
 
+  if job_data["@argv"]
+    argv = job_data["@argv"]
+    if argv.class != String and argv.class != Array
+      abort("The value specified for :argv must be either a String or Array") 
+    end
+
+    if argv.class == Array
+      job_data["@argv"] = argv.join(' ')
+    end
+  end
+
   return job_data
 end
 

data/test/unit/test_neptune.rb

@@ -81,17 +81,46 @@ class TestNeptune < Test::Unit::TestCase
   end
 
   def test_preprocess_mpi
+    # not specifying nodes to use or procs to use should throw an error
     job_data_1 = {}
     assert_raise(SystemExit) { preprocess_mpi(job_data_1) }
 
+    # not specifying procs to use should throw an error
     job_data_2 = {"@nodes_to_use" => 4}
     assert_raise(SystemExit) { preprocess_mpi(job_data_2) }
 
+    # specifying procs to use == nodes to use should not throw an error
     job_data_3 = {"@nodes_to_use" => 4, "@procs_to_use" => 4}
     assert_equal(job_data_3, preprocess_mpi(job_data_3))
 
+    # specifying procs to use < nodes to use should throw an error
     job_data_4 = {"@nodes_to_use" => 4, "@procs_to_use" => 1}
     assert_raise(SystemExit) { preprocess_mpi(job_data_4) }
+ 
+    # specifying an empty string for argv should be ok
+    job_data_5 = {"@nodes_to_use" => 4, "@procs_to_use" => 4, "@argv" => ""}
+    assert_equal(job_data_5, preprocess_mpi(job_data_5))
+
+    # specifying an empty array for argv should be ok
+    job_data_6 = {"@nodes_to_use" => 4, "@procs_to_use" => 4, "@argv" => []}
+    expected_job_data_6 = job_data_6.dup
+    expected_job_data_6["@argv"] = ""
+    assert_equal(expected_job_data_6, preprocess_mpi(job_data_6))
+
+    # specifying something that's not a string or array for argv should throw
+    # an error
+    job_data_7 = {"@nodes_to_use" => 4, "@procs_to_use" => 4, "@argv" => 2}
+    assert_raise(SystemExit) { preprocess_mpi(job_data_7) }
+
+    # specifying a non-empty string for argv should be ok
+    job_data_8 = {"@nodes_to_use" => 4, "@procs_to_use" => 4, "@argv" => "--file coo"}
+    assert_equal(job_data_8, preprocess_mpi(job_data_8))
+
+    # specifying a non-empty array for argv should be ok
+    job_data_9 = {"@nodes_to_use" => 4, "@procs_to_use" => 4, "@argv" => ["--file", "coo"]}
+    expected_job_data_9 = job_data_9.dup
+    expected_job_data_9["@argv"] = "--file coo"
+    assert_equal(expected_job_data_9, preprocess_mpi(job_data_9))
   end
 
   def test_preprocess_ssa

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification 
 name: neptune
 version: !ruby/object:Gem::Version 
-  hash: 29
+  hash: 19
   prerelease: 
   segments: 
   - 0
   - 1
-  - 3
-  version: 0.1.3
+  - 4
+  version: 0.1.4
 platform: ruby
 authors: 
 - Chris Bunch
@@ -15,7 +15,7 @@ autorequire: neptune
 bindir: bin
 cert_chain: []
 
-date: 2011-12-04 00:00:00 -08:00
+date: 2011-12-06 00:00:00 -08:00
 default_executable: neptune
 dependencies: []