org-converge 0.0.6 → 0.0.7

data/TODO

@@ -2,19 +2,28 @@
 #+TODO:    TODO | DONE CANCELED 
 #+startup: showeverything
 
-* [0/8] > 0.0.9 version
+* [0/7] > 0.0.9 version
 
 - [ ] Macros can be loaded and applied to the configuration
 - [ ] Actually support converging and idempotency
   + Do not do an operation unless it is required
   + Abort in case there was a failure in executing the script.
-- [ ] Support SETUPFILE
+  + This would work by using the ~:cache true~ to a block header argument
+- [ ] Support for ~#+SETUPFILE~
 - [ ] Heuristics for determining which binary to use for running the script
 - [ ] Display how the run would look like without making changes
   : org-converge setupfile.org --dry-run
 - [ ] Use :eval for evaling blocks (off by default)
 - [ ] Can use :dir for running a process relative to that directory
-- [ ] Use rake task manager
+
+* [0/1] 0.0.8 version
+
+- [ ] ~org-spec~ exploration
+
+* [2/2] 0.0.7 version
+
+- [X] Add ~org-run~ and ~org-tangle~ aliases
+- [X] Use rake task manager
 
 * [3/3] 0.0.6 version
 
@@ -55,37 +64,36 @@ Need some basic functionality of what Org babel offers first.
 - [X] ~--tangle~ flag
 - [X] Support a root dir for when not running relative to the directory
 
-* [3/20] Ideas
-** CANCELED How to set the permissions from the directory from the file that is being tangled when it does not exists?
+* [6/21] Ideas
+** TODO Chaining resources with ~#+NAME:~ directives and ~:notify~ argument
 
-By default, this would be 0644, but we also need to specify the
-user:group of it so the syntax would have to be:
+One idea is to be able to notify resources by naming the code blocks.
 
-#+begin_src conf :tangle etc/something/config/path :chmod 0664 :user td-agent :group
-hmm dont't like this syntax for folders
+Example: Here first the td-agent service would try to start,
+and if it succeeds, then it would execute the script defined in the
+~announce-availability~ resource.
+
+#+name: td-agent-start
+#+begin_src sh :notify announce-availability
+sudo service td-agent start
 #+end_src
 
-Let's keep it simple and just use a babel block that shells out to create the directories
-until I can think of something better.
+#+name: announce-availability
+#+begin_src sh
+sudo /etc/register-to-balancer
+#+end_src
+
+- Support for ~:before all~ and ~:after all~ added
+** TODO Add namespace dynamically to the tasks using an in buffer setting
+
+When including files there can be some potential scoping issues.
+
 ** TODO Inject macros within the environment variables from a runnable process
 ** TODO Clarify which ones of the header arguments to implement
 
 http://orgmode.org/manual/Specific-header-arguments.html#Specific-header-arguments
 
 ** TODO Use sshkit for running remote processes
-** TODO We don't need to create the directories in most cases (:mkdirp yes)
-
-Something like this is not required because the ~:tangle~ blocks
-would create the necessary directories behind the scenes.
-
-#+begin_src org
-  ,We need to prepare some directories for the configuration:
-  
-  ,#+begin_src converge
-  ,mkdir -p etc/fluentd/config
-  ,#+end_src
-#+end_src
-
 ** TODO By default, it should use current dir for tangling
 ** TODO Converging: Only do an operation when it didn't finish
 
@@ -130,24 +138,26 @@ fluentd:
 
 But need to 
 
-** TODO Using the :tags: to setup the things to run right away
+** TODO Choosing a templating language: default for now is mustache
 
-Kind of like the chef-solo runlist, a headline like this...
+We could implement the macro systems, but it seems that it may not be
+strong enough for providing with all the cases we may run into.
+** TODO Support caching?
 
-#+begin_src org
-  ,* Everything in its right place                                      :config:
-  
-  ,#+begin_src conf :tangle etc/this.yml
-  ,hello: "world"
-  ,#+end_src
+#+begin_src emacs-lisp :cache yes :exports results
+  (random)
 #+end_src
 
-...could be called like this
+#+RESULTS[db54597aed193d861d01bf92110e10f28f8f40d4]:
+: 842438499349743708
 
-#+begin_src sh
-org-converge fluentd.org -t config
+** TODO Support :eval ?
+
+#+begin_src sh :eval (print "Really doing this...")
+  echo "Going ahead with operation X!"
 #+end_src
 
+** DONE ~#+NAME:~ could be used in the logger for identifying the process
 ** DONE Managing dependencies: could be handled with ~#+include~ directives
 
 One idea is that it would be useful to compile different manuals
@@ -167,52 +177,58 @@ We should be able to express that dependency somehow:
 
 # But one problem, is that once I have included something, sometimes we
 # will want "reopen" the previous manuals?
+** DONE We don't need to create the directories in most cases (:mkdirp yes)
 
-** TODO Loading all the Org mode files first and then setup only one part
+Something like this is not required because the ~:tangle~ blocks
+would create the necessary directories behind the scenes.
 
-So maybe, each one of these tags would have to be namespaces under the
-name of the file:
+#+begin_src org
+  ,We need to prepare some directories for the configuration:
+  
+  ,#+begin_src converge
+  ,mkdir -p etc/fluentd/config
+  ,#+end_src
+#+end_src
+** CANCELED How to set the permissions from the directory from the file that is being tangled when it does not exists?
 
-: org-converge logserver.org -t "fluentd::setup, fluentd::config, logserver::setup"
+By default, this would be 0644, but we also need to specify the
+user:group of it so the syntax would have to be:
 
-** TODO Choosing a templating language: default for now is mustache
+#+begin_src conf :tangle etc/something/config/path :chmod 0664 :user td-agent :group
+hmm dont't like this syntax for folders
+#+end_src
 
-We could implement the macro systems, but it seems that it may not be
-strong enough for providing with all the cases we may run into.
+Let's keep it simple and just use a babel block that shells out to create the directories
+until I can think of something better.
 
-** TODO Chaining resources with ~#+NAME:~ directives and ~:notify~ argument
+** CANCELED Using the :tags: to setup the things to run right away
 
-One idea is to be able to notify resources by naming the code blocks.
+Using the block ~#+name~ instead.
 
-Example: Here first the td-agent service would try to start,
-and if it succeeds, then it would execute the script defined in the
-~announce-availability~ resource.
+Kind of like the chef-solo runlist, a headline like this...
 
-#+name: td-agent-start
-#+begin_src sh :notify announce-availability
-sudo service td-agent start
+#+begin_src org
+  ,* Everything in its right place                                      :config:
+  
+  ,#+begin_src conf :tangle etc/this.yml
+  ,hello: "world"
+  ,#+end_src
 #+end_src
 
-#+name: announce-availability
+...could be called like this
+
 #+begin_src sh
-sudo /etc/register-to-balancer
+org-converge fluentd.org -t config
 #+end_src
 
-** DONE ~#+NAME:~ could be used in the logger for identifying the process
-** TODO Support caching?
-
-#+begin_src emacs-lisp :cache yes :exports results
-  (random)
-#+end_src
+** CANCELED Loading all the Org mode files first and then setup only one part
 
-#+RESULTS[db54597aed193d861d01bf92110e10f28f8f40d4]:
-: 842438499349743708
+Used ~#+name~ from blocks instead.
 
-** TODO Support :eval ?
+So maybe, each one of these tags would have to be namespaces under the
+name of the file:
 
-#+begin_src sh :eval (print "Really doing this...")
-  echo "Going ahead with operation X!"
-#+end_src
+: org-converge logserver.org -t "fluentd::setup, fluentd::config, logserver::setup"
 
 * Links
 

data/bin/org-converge

@@ -8,7 +8,7 @@ doc = <<OPTIONS
 org-converge: A light configuration management tool for Org mode
 
 Usage:
-  org-converge <org_file> [--tangle] [--showfiles] [--log=<logfile>] [--root-dir=<root_dir>] [--runmode=<runmode>] [--name=<block_name>]
+  org-converge <org_file> [--showfiles] [--log=<logfile>] [--root-dir=<root_dir>] [--runmode=<runmode>] [--name=<block_name>]
 
 Options:
 

data/bin/org-run

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+# -*- mode: ruby -*-
+
+# alias for org-converge
+load File.expand_path('../org-converge', __FILE__)

data/bin/org-tangle

@@ -0,0 +1,29 @@
+#!/usr/bin/env ruby
+# -*- mode: ruby -*-
+require 'docopt'
+require 'org-converge'
+
+doc = <<OPTIONS
+
+org-tangle: Org babel tangling feature using the Org mode Ruby parser
+
+Usage:
+  org-tangle <org_file> [--showfiles] [--log=<logfile>] [--root-dir=<root_dir>]
+
+Options:
+
+  -h --help               Show this screen.
+
+OPTIONS
+
+begin
+  require "pp"
+  cmd = Docopt::docopt(doc)
+rescue Docopt::Exit => e
+  puts e.message
+end
+
+exit 1 unless cmd
+
+o = OrgConverge::Command.new(cmd.merge({'--tangle' => true }))
+o.execute!

data/lib/org-converge/babel_output_buffer.rb

@@ -1,3 +1,5 @@
+require 'rake'
+
 module Orgmode
   class BabelOutputBuffer < OutputBuffer
     attr_reader :tangle
@@ -11,11 +13,21 @@ module Orgmode
 
       # ~@tangle~ files are put in the right path
       # : @tangle['/path'] = [Lines]
-      @tangle  = Hash.new {|h,k| h[k] = {:lines => '', :header => {}, :lang => ''}}
+      @tangle  = Hash.new {|h,k| h[k] = {
+          :lines  => '',
+          :header => {},
+          :lang   => ''
+        }
+      }
 
       # ~@scripts~ are tangled in order and ran
       # : @scripts = [text, text, ...]
-      @scripts = Hash.new {|h,k| h[k] = {:lines => '', :header => {}, :lang => ''}}
+      @scripts = Hash.new {|h,k| h[k] = {
+          :lines  => '',
+          :header => {},
+          :lang   => ''
+        }
+      }
       @scripts_counter = 0
       @buffer = ''
     end
@@ -50,7 +62,9 @@ module Orgmode
           @scripts[@scripts_counter][:header] = { 
             :shebang => line.block_header_arguments[':shebang'],
             :mkdirp  => line.block_header_arguments[':mkdirp'],
-            :name    => line.properties['block_name']
+            :name    => line.properties['block_name'],
+            :before  => line.block_header_arguments[':before'],
+            :after   => line.block_header_arguments[':after']
           }
           @scripts[@scripts_counter][:lang] = line.block_lang
         # TODO: have a way to specify which are the default binaries to be used per language

data/lib/org-converge/command.rb

@@ -58,8 +58,8 @@ module OrgConverge
         run_blocks_in_parallel!
       when 'sequential'
         run_blocks_sequentially!
-      when 'runlist'
-        # TODO
+      when 'chained', 'chain', 'tasks'
+        run_blocks_chain!
       else # parallel by default
         run_blocks_in_parallel!
       end
@@ -71,6 +71,53 @@ module OrgConverge
       logger.error "Cannot converge because there were errors during tangle step".fg 'red'
     end
 
+    def run_blocks_chain!
+      # Chain the blocks by defining them as Rake::Tasks dynamically
+      tasks = { }
+
+      babel.tangle_runnable_blocks!(@run_dir)
+      babel.ob.scripts.each do |key, script|
+        task_name = script[:header][:name]
+        next unless task_name
+
+        task = Rake::Task.define_task task_name do
+          with_running_engine do |engine|
+            file = File.expand_path("#{@run_dir}/#{key}")
+            cmd = "#{script[:lang]} #{file}"
+            engine.register task_name, cmd, { :cwd => @root_dir, :logger => logger }
+          end
+        end
+        tasks[task_name] = { 
+          :task => task,
+          :script => script
+        }
+      end
+
+      # Now onto define the prerequisites and actions
+      tasks.each_pair do |task_name, task_definition|
+        prerequisite_task = task_definition[:script][:header][:after]
+        if prerequisite_task and tasks[prerequisite_task]
+          task_definition[:task].prerequisites << tasks[prerequisite_task][:task]
+        end
+
+        postrequisite_task = task_definition[:script][:header][:before]
+        if postrequisite_task and tasks[postrequisite_task]
+          tasks[postrequisite_task][:task].prerequisites << task_definition[:task]
+        end
+      end
+
+      # The task that marks the run as done needs to be defined explicitly
+      # otherwise a block named default will tried to be run
+      final_task = babel.ob.in_buffer_settings['FINAL_TASK'] || 'default'
+
+      if tasks[final_task]
+        logger.info "Running final task: #{tasks[final_task][:task]}"
+        tasks[final_task][:task].invoke
+      else
+        logger.error "Could not find a final task to run!"
+      end
+    end
+
     def run_blocks_sequentially!
       @engine = OrgConverge::Engine.new(:logger => @logger, :babel => @babel)
       babel.tangle_runnable_blocks!(@run_dir)
@@ -96,8 +143,6 @@ module OrgConverge
       logger.info "Run has completed successfully.".fg 'green'
     end
 
-    # TODO: Too much foreman has made this running blocks in parallel the default behavior.
-    #       We should actually be supporting run lists instead, but liking this experiment so far.
     def run_blocks_in_parallel!
       @engine = OrgConverge::Engine.new(:logger => @logger, :babel => @babel)
       babel.tangle_runnable_blocks!(@run_dir)

data/lib/org-converge/version.rb

@@ -1,3 +1,3 @@
 module OrgConverge
-  VERSION = "0.0.6"
+  VERSION = "0.0.7"
 end

data/org-converge.gemspec

@@ -14,8 +14,9 @@ Gem::Specification.new do |gem|
   gem.name          = "org-converge"
   gem.require_paths = ["lib"]
   gem.version       = OrgConverge::VERSION
-  gem.add_runtime_dependency('docopt', '0.5.0')
-  gem.add_runtime_dependency('org-ruby', '~> 0.9.4')
-  gem.add_runtime_dependency('foreman', '~> 0.63.0')
-  gem.add_runtime_dependency('tco', '~> 0.1.0')
+  gem.add_runtime_dependency('docopt',   '~> 0.5.0')
+  gem.add_runtime_dependency('org-ruby', '~> 0.9.5')
+  gem.add_runtime_dependency('foreman',  '~> 0.63.0')
+  gem.add_runtime_dependency('tco',      '~> 0.1.0')
+  gem.add_runtime_dependency('rake',     '~> 10.3')
 end

data/spec/converge_examples/linked_tasks/tasks.org

@@ -0,0 +1,43 @@
+#+TITLE:           Linked tasks example
+#+runmode:         tasks
+#+final_task:      final
+
+The goal here is to be able to define dependencies
+among blocks and run the correctly in that order.
+
+With ~:after first~, the ~second~ block becomes
+a prerequisite of the ~first~ block.
+
+#+name: second
+#+begin_src sh :after first
+for i in `seq 5 10`; do 
+  echo $i >> out.log
+done
+#+end_src
+
+#+name: first
+#+begin_src ruby
+5.times { |n| File.open("out.log", "a") {|f| f.puts n } }
+#+end_src
+
+Using ~:after all~ is special, since it means that the task
+depends on everything else having run already.
+
+#+name: final
+#+begin_src python :after second :results output
+print "Wrapping up with Python in the end"
+f = open('out.log', 'a')
+f.write('11')
+f.close()
+#+end_src
+
+The same with ~:before all~, this would be executed
+at the beginning.
+
+#+name: prologue
+#+begin_src sh :before first :results output
+echo "init" > out.log
+#+end_src
+
+Note: Both ~:before all~ and ~:after all~ get accumulated
+and are run sequentially after that.

data/spec/converge_spec.rb

@@ -85,4 +85,24 @@ describe OrgConverge::Command do
     expected_contents = "first\nsecond\n"
     File.read(File.join(example_dir, 'out.log')).should == expected_contents
   end
+
+  it "should run 'linked_tasks' in order" do
+    example_dir = File.join(EXAMPLES_DIR, 'linked_tasks')
+    setup_file = File.join(example_dir, 'tasks.org')
+
+    o = OrgConverge::Command.new({ 
+                                   '<org_file>' => setup_file,
+                                   '--root-dir' => example_dir
+                                 })
+    success = o.execute!
+    success.should == true
+
+    File.executable?(File.join(example_dir, 'run/0')).should == true
+    File.executable?(File.join(example_dir, 'run/1')).should == true
+    File.executable?(File.join(example_dir, 'run/2')).should == true
+    File.executable?(File.join(example_dir, 'run/3')).should == true
+
+    expected_contents = "init\n0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11"
+    File.read(File.join(example_dir, 'out.log')).should == expected_contents
+  end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: org-converge
 version: !ruby/object:Gem::Version
-  version: 0.0.6
+  version: 0.0.7
   prerelease: 
 platform: ruby
 authors:
@@ -9,33 +9,33 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-05-03 00:00:00.000000000 Z
+date: 2014-05-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: docopt
-  requirement: &70130002051260 !ruby/object:Gem::Requirement
+  requirement: &70254668243000 !ruby/object:Gem::Requirement
     none: false
     requirements:
-    - - =
+    - - ~>
       - !ruby/object:Gem::Version
         version: 0.5.0
   type: :runtime
   prerelease: false
-  version_requirements: *70130002051260
+  version_requirements: *70254668243000
 - !ruby/object:Gem::Dependency
   name: org-ruby
-  requirement: &70130002050780 !ruby/object:Gem::Requirement
+  requirement: &70254668242520 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
       - !ruby/object:Gem::Version
-        version: 0.9.4
+        version: 0.9.5
   type: :runtime
   prerelease: false
-  version_requirements: *70130002050780
+  version_requirements: *70254668242520
 - !ruby/object:Gem::Dependency
   name: foreman
-  requirement: &70130002050320 !ruby/object:Gem::Requirement
+  requirement: &70254664800460 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -43,10 +43,10 @@ dependencies:
         version: 0.63.0
   type: :runtime
   prerelease: false
-  version_requirements: *70130002050320
+  version_requirements: *70254664800460
 - !ruby/object:Gem::Dependency
   name: tco
-  requirement: &70130002049860 !ruby/object:Gem::Requirement
+  requirement: &70254664799380 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ~>
@@ -54,12 +54,25 @@ dependencies:
         version: 0.1.0
   type: :runtime
   prerelease: false
-  version_requirements: *70130002049860
+  version_requirements: *70254664799380
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: &70254664798160 !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: '10.3'
+  type: :runtime
+  prerelease: false
+  version_requirements: *70254664798160
 description: A light configuration management tool for Org mode
 email:
 - waldemar.quevedo@gmail.com
 executables:
 - org-converge
+- org-run
+- org-tangle
 extensions: []
 extra_rdoc_files: []
 files:
@@ -71,6 +84,8 @@ files:
 - Rakefile
 - TODO
 - bin/org-converge
+- bin/org-run
+- bin/org-tangle
 - examples/apt-get-install/setup.org
 - examples/fluentd/setup.org
 - examples/macro-config/setup.org
@@ -86,6 +101,7 @@ files:
 - spec/converge_examples/basic_tangle/conf.yml.expected
 - spec/converge_examples/basic_tangle/setup.org
 - spec/converge_examples/commented_block/run.org
+- spec/converge_examples/linked_tasks/tasks.org
 - spec/converge_examples/runlist_example/setup.org
 - spec/converge_examples/specified_block/run.org
 - spec/converge_spec.rb
@@ -120,6 +136,7 @@ test_files:
 - spec/converge_examples/basic_tangle/conf.yml.expected
 - spec/converge_examples/basic_tangle/setup.org
 - spec/converge_examples/commented_block/run.org
+- spec/converge_examples/linked_tasks/tasks.org
 - spec/converge_examples/runlist_example/setup.org
 - spec/converge_examples/specified_block/run.org
 - spec/converge_spec.rb