org-converge 0.0.4 → 0.0.5
Sign up to get free protection for your applications and to get access to all the features.
- data/README.org +112 -62
- data/TODO +20 -6
- data/examples/simple/setup.org +0 -9
- data/lib/org-converge/babel_output_buffer.rb +12 -7
- data/lib/org-converge/command.rb +27 -9
- data/lib/org-converge/version.rb +1 -1
- data/lib/org-converge.rb +9 -18
- data/org-converge.gemspec +1 -1
- data/spec/converge_examples/basic_run_example/setup.org +29 -17
- data/spec/converge_examples/runlist_example/setup.org +4 -1
- data/spec/converge_spec.rb +1 -2
- metadata +11 -12
- data/README.md +0 -157
data/README.org
CHANGED
@@ -1,15 +1,15 @@
|
|
1
1
|
# -*- mode: org; mode: auto-fill; -*-
|
2
|
-
#+TITLE: Org Converge
|
3
2
|
#+STARTUP: showeverything
|
4
3
|
|
5
|
-
*
|
4
|
+
* Org Converge
|
5
|
+
** Description
|
6
6
|
|
7
7
|
This attempts to be an experiment of using Org mode syntax to
|
8
8
|
describe, configure and setting up something, borrowing some ideas
|
9
9
|
of what is possible to do with tools like =chef-solo=, =puppet=,
|
10
10
|
=ansible=, etc...
|
11
11
|
|
12
|
-
|
12
|
+
** Motivation
|
13
13
|
|
14
14
|
The Org babel syntax has proven to be flexible enough to produce
|
15
15
|
/reproducible research/ papers. Then, I believe that configuring and setting up
|
@@ -17,11 +17,84 @@
|
|
17
17
|
the same syntax, given that /converging/ the configuration is something
|
18
18
|
that one ought to be able to reproduce.
|
19
19
|
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
20
|
+
** Usage
|
21
|
+
|
22
|
+
To run the blocks in parallel:
|
23
|
+
|
24
|
+
#+begin_src sh
|
25
|
+
org-converge path/to/setup-file.org
|
26
|
+
#+end_src
|
27
|
+
|
28
|
+
Or sequentially:
|
29
|
+
|
30
|
+
#+begin_src sh
|
31
|
+
org-converge path/to/setup-file.org --runmode=sequential
|
32
|
+
#+end_src
|
33
|
+
|
34
|
+
** Quick example
|
35
|
+
|
36
|
+
The following is an example of running 3 processes
|
37
|
+
in parallel by defining them as code blocks from
|
38
|
+
an Org mode file:
|
39
|
+
|
40
|
+
#+begin_src sh
|
41
|
+
,#+TITLE: Running Org babel processes in parallel
|
42
|
+
|
43
|
+
* Print with different languages
|
44
|
+
|
45
|
+
,#+name: hello_from_bash
|
46
|
+
,#+begin_src sh :shebang #!/bin/bash
|
47
|
+
while true; do echo "hello world from bash"; sleep 1; done
|
48
|
+
,#+end_src
|
49
|
+
|
50
|
+
,#+name: hello_from_ruby
|
51
|
+
,#+begin_src ruby :shebang #!/usr/local/bin/ruby
|
52
|
+
$stdout.sync = true
|
53
|
+
loop { puts "hello world from ruby" ; sleep 1 }
|
54
|
+
,#+end_src
|
55
|
+
|
56
|
+
,#+name: hello_from_python
|
57
|
+
,#+begin_src python :shebang #!/usr/bin/python
|
58
|
+
import time
|
59
|
+
import sys
|
60
|
+
for i in range(0,100):
|
61
|
+
print "hello world from python"
|
62
|
+
sys.stdout.flush()
|
63
|
+
time.sleep(1)
|
64
|
+
,#+end_src
|
65
|
+
#+end_src
|
66
|
+
|
67
|
+
We store this in a file named =hello.org= and then run it as follows:
|
68
|
+
|
69
|
+
#+begin_src sh
|
70
|
+
org-converge hello.org
|
71
|
+
#+end_src
|
72
|
+
|
73
|
+
This would produce an output similar to the following:
|
74
|
+
|
75
|
+
#+begin_src sh
|
76
|
+
I, [2014-04-17T23:48:44.663545 #1710] INFO -- : Tangling 0 files...
|
77
|
+
I, [2014-04-17T23:48:44.663698 #1710] INFO -- : Tangling succeeded!
|
78
|
+
I, [2014-04-17T23:48:44.664829 #1710] INFO -- : Running code blocks now! (4 runnable blocks found in total)
|
79
|
+
I, [2014-04-17T23:48:44.762899 #1710] INFO -- : hello_from_bash (1711) -- started with pid 1711
|
80
|
+
I, [2014-04-17T23:48:44.808717 #1710] INFO -- : hello_from_ruby (1712) -- started with pid 1712
|
81
|
+
I, [2014-04-17T23:48:44.837878 #1710] INFO -- : hello_from_python (1713) -- started with pid 1713
|
82
|
+
I, [2014-04-17T23:48:44.878336 #1710] INFO -- : (1714) -- started with pid 1714
|
83
|
+
I, [2014-04-17T23:48:44.879284 #1710] INFO -- : hello_from_bash (1711) -- hello world from bash
|
84
|
+
I, [2014-04-17T23:48:44.882778 #1710] INFO -- : hello_from_ruby (1712) -- hello world from ruby
|
85
|
+
I, [2014-04-17T23:48:44.883637 #1710] INFO -- : hello_from_python (1713) -- hello world from python
|
86
|
+
I, [2014-04-17T23:48:45.778681 #1710] INFO -- : hello_from_bash (1711) -- hello world from bash
|
87
|
+
I, [2014-04-17T23:48:45.792321 #1710] INFO -- : hello_from_ruby (1712) -- hello world from ruby
|
88
|
+
I, [2014-04-17T23:48:45.882020 #1710] INFO -- : hello_from_python (1713) -- hello world from python
|
89
|
+
I, [2014-04-17T23:48:46.787106 #1710] INFO -- : hello_from_bash (1711) -- hello world from bash
|
90
|
+
I, [2014-04-17T23:48:46.793347 #1710] INFO -- : hello_from_ruby (1712) -- hello world from ruby
|
91
|
+
I, [2014-04-17T23:48:46.883511 #1710] INFO -- : hello_from_python (1713) -- hello world from python
|
92
|
+
I, [2014-04-17T23:48:47.794846 #1710] INFO -- : hello_from_ruby (1712) -- hello world from ruby
|
93
|
+
I, [2014-04-17T23:48:47.796387 #1710] INFO -- : hello_from_bash (1711) -- hello world from bash
|
94
|
+
I, [2014-04-17T23:48:47.884196 #1710] INFO -- : hello_from_python (1713) -- hello world from python
|
95
|
+
#+end_src
|
96
|
+
|
97
|
+
** How it works
|
25
98
|
|
26
99
|
Org Converge uses an liberally extended version of Org Babel
|
27
100
|
features in order to give support for converging the configuration
|
@@ -30,10 +103,7 @@
|
|
30
103
|
For example, using Org Babel and macros we can easily spread config
|
31
104
|
files on a server by writing the following on a ~server.org~ file.
|
32
105
|
|
33
|
-
#+begin_src
|
34
|
-
,#+MACRO: multitenancy_enabled true
|
35
|
-
,#+MACRO: status_port true
|
36
|
-
|
106
|
+
#+begin_src sh
|
37
107
|
,#+begin_src yaml :tangle /etc/component.yml
|
38
108
|
multitenant: false
|
39
109
|
status_port: 10004
|
@@ -41,69 +111,49 @@
|
|
41
111
|
#+end_src
|
42
112
|
|
43
113
|
And then configure it by running it as follows, (considering we have
|
44
|
-
the correct permissions):
|
45
|
-
|
46
|
-
#+begin_src sh
|
47
|
-
org-converge server.org
|
48
|
-
#+end_src
|
49
|
-
|
50
|
-
This leverages on the syntax already provided by Org Babel, but one
|
51
|
-
difference here is that if we run it once again without changes...
|
114
|
+
the correct permissions for tangling at =/etc/component.yml=):
|
52
115
|
|
53
116
|
#+begin_src sh
|
54
|
-
org-converge server.org
|
117
|
+
sudo org-converge server.org
|
55
118
|
#+end_src
|
56
|
-
|
57
|
-
...it would finish soon since the configuration has already converged.
|
58
119
|
|
59
120
|
Next, let's say that we no only one want to set the configured templates,
|
60
121
|
but that we also want to install some packages. In that case, we
|
61
122
|
should be able to do the following:
|
62
123
|
|
63
|
-
#+begin_src
|
64
|
-
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
|
70
|
-
|
71
|
-
|
72
|
-
|
73
|
-
|
74
|
-
|
75
|
-
|
76
|
-
|
77
|
-
|
78
|
-
|
79
|
-
|
80
|
-
|
81
|
-
|
82
|
-
|
83
|
-
|
84
|
-
|
85
|
-
|
86
|
-
,#+begin_src sh
|
87
|
-
cd {{{project_path}}}
|
88
|
-
bundle install
|
89
|
-
,#+end_src
|
124
|
+
#+begin_src sh
|
125
|
+
,* Configuring the component
|
126
|
+
|
127
|
+
,#+begin_src yaml :tangle /etc/component.yml
|
128
|
+
multitenant: false
|
129
|
+
status_port: 10004
|
130
|
+
,#+end_src
|
131
|
+
|
132
|
+
,* Installing the dependencies
|
133
|
+
|
134
|
+
Need the following so that ~bundle install~ can compile
|
135
|
+
the native extensions correctly.
|
136
|
+
|
137
|
+
,#+begin_src sh
|
138
|
+
apt-get install build-essentials -y
|
139
|
+
,#+end_src
|
140
|
+
|
141
|
+
Then the following should work:
|
142
|
+
|
143
|
+
,#+begin_src sh
|
144
|
+
cd project_path
|
145
|
+
bundle install
|
146
|
+
,#+end_src
|
90
147
|
#+end_src
|
91
148
|
|
92
149
|
As long as the repo has been already checked out in the directory,
|
93
150
|
the previous example will succeed.
|
94
151
|
|
95
|
-
|
96
|
-
org-converge server.org
|
97
|
-
#+end_src
|
98
|
-
|
99
|
-
If that is not the case, then org-converge will fail
|
100
|
-
and pickup from that last step.
|
101
|
-
|
102
|
-
More practical examples can be found [[here]], more will be added as
|
152
|
+
More practical examples can be found [[https://github.com/wallyqs/org-converge/tree/master/examples][here]], more will be added as
|
103
153
|
long as dogfooding from this goes well.
|
104
154
|
|
105
|
-
|
155
|
+
** Contributing
|
106
156
|
|
107
|
-
The project is in very early development at this moment, but if you
|
108
|
-
feel that it is interesting enough, please create a ticket so start
|
109
|
-
the discussion.
|
157
|
+
The project is in very early development at this moment, but if you
|
158
|
+
feel that it is interesting enough, please create a ticket so start
|
159
|
+
the discussion.
|
data/TODO
CHANGED
@@ -2,9 +2,8 @@
|
|
2
2
|
#+TODO: TODO | DONE CANCELED
|
3
3
|
#+startup: showeverything
|
4
4
|
|
5
|
-
* [0/
|
5
|
+
* [0/7] > 0.0.9 version
|
6
6
|
|
7
|
-
- [ ] Support runlists notifications via =#+NAME:=
|
8
7
|
- [ ] Macros can be loaded and applied to the configuration
|
9
8
|
- [ ] Actually support converging and idempotency
|
10
9
|
+ Do not do an operation unless it is required
|
@@ -16,6 +15,21 @@
|
|
16
15
|
- [ ] Use :eval for evaling blocks (off by default)
|
17
16
|
- [ ] Can use :dir for running a process relative to that directory
|
18
17
|
|
18
|
+
* [0/3] 0.0.6 version
|
19
|
+
|
20
|
+
- [ ] Keyword to accumulate run blocks
|
21
|
+
- [ ] Do not run commented blocks
|
22
|
+
- [ ] Use rake task manager
|
23
|
+
|
24
|
+
* [2/2] 0.0.5 version
|
25
|
+
|
26
|
+
- [X] Do not run or tangle blocks without arguments
|
27
|
+
- [X] Keyword to activate runmode in the text itself
|
28
|
+
|
29
|
+
* [1/1] 0.0.4 version
|
30
|
+
|
31
|
+
- [X] Support runlists notifications via =#+NAME:=
|
32
|
+
|
19
33
|
* [2/2] 0.0.3 version
|
20
34
|
|
21
35
|
- [X] Support sequential and parallel execution
|
@@ -39,7 +53,7 @@ Need some basic functionality of what Org babel offers first.
|
|
39
53
|
- [X] ~--tangle~ flag
|
40
54
|
- [X] Support a root dir for when not running relative to the directory
|
41
55
|
|
42
|
-
* [
|
56
|
+
* [3/19] Ideas
|
43
57
|
** CANCELED How to set the permissions from the directory from the file that is being tangled when it does not exists?
|
44
58
|
|
45
59
|
By default, this would be 0644, but we also need to specify the
|
@@ -131,7 +145,7 @@ Kind of like the chef-solo runlist, a headline like this...
|
|
131
145
|
org-converge fluentd.org -t config
|
132
146
|
#+end_src
|
133
147
|
|
134
|
-
**
|
148
|
+
** DONE Managing dependencies: could be handled with ~#+include~ directives
|
135
149
|
|
136
150
|
One idea is that it would be useful to compile different manuals
|
137
151
|
that are dependent among each other.
|
@@ -181,7 +195,7 @@ sudo service td-agent start
|
|
181
195
|
sudo /etc/register-to-balancer
|
182
196
|
#+end_src
|
183
197
|
|
184
|
-
**
|
198
|
+
** DONE ~#+NAME:~ could be used in the logger for identifying the process
|
185
199
|
** TODO Support caching?
|
186
200
|
|
187
201
|
#+begin_src emacs-lisp :cache yes :exports results
|
@@ -199,6 +213,6 @@ sudo /etc/register-to-balancer
|
|
199
213
|
|
200
214
|
* Links
|
201
215
|
|
202
|
-
Super useful notes:
|
203
216
|
http://eschulte.github.io/org-scraps
|
204
217
|
|
218
|
+
http://doc.norang.ca/org-mode.html
|
data/examples/simple/setup.org
CHANGED
@@ -7,20 +7,11 @@ It does not get more simple than this:
|
|
7
7
|
#+begin_src conf :tangle /tmp/my-conf
|
8
8
|
<source>
|
9
9
|
bind 0.0.0.0
|
10
|
-
|
11
|
-
|
12
|
-
|
13
10
|
port 4224
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
11
|
</source>
|
19
12
|
|
20
13
|
<match **>
|
21
14
|
type file
|
22
|
-
|
23
|
-
|
24
15
|
path /var/tmp/a.log
|
25
16
|
</match>
|
26
17
|
#+end_src
|
@@ -2,10 +2,13 @@ module Orgmode
|
|
2
2
|
class BabelOutputBuffer < OutputBuffer
|
3
3
|
attr_reader :tangle
|
4
4
|
attr_reader :scripts
|
5
|
+
attr_reader :in_buffer_settings
|
5
6
|
|
6
|
-
def initialize(output)
|
7
|
+
def initialize(output, opts={})
|
7
8
|
super(output)
|
8
9
|
|
10
|
+
@in_buffer_settings = opts[:in_buffer_settings]
|
11
|
+
|
9
12
|
# ~@tangle~ files are put in the right path
|
10
13
|
# : @tangle['/path'] = [Lines]
|
11
14
|
@tangle = Hash.new {|h,k| h[k] = {:lines => '', :header => {}, :lang => ''}}
|
@@ -14,7 +17,6 @@ module Orgmode
|
|
14
17
|
# : @scripts = [text, text, ...]
|
15
18
|
@scripts = Hash.new {|h,k| h[k] = {:lines => '', :header => {}, :lang => ''}}
|
16
19
|
@scripts_counter = 0
|
17
|
-
|
18
20
|
@buffer = ''
|
19
21
|
end
|
20
22
|
|
@@ -29,7 +31,7 @@ module Orgmode
|
|
29
31
|
end
|
30
32
|
|
31
33
|
def insert(line)
|
32
|
-
# We try to get the lang from #+BEGIN_SRC blocks
|
34
|
+
# We try to get the lang from #+BEGIN_SRC and #+BEGIN_EXAMPLE blocks
|
33
35
|
if line.begin_block?
|
34
36
|
case
|
35
37
|
when line.block_header_arguments[':tangle']
|
@@ -39,7 +41,8 @@ module Orgmode
|
|
39
41
|
:mkdirp => line.block_header_arguments[':mkdirp']
|
40
42
|
}
|
41
43
|
@tangle[@current_tangle][:lang] = line.block_lang
|
42
|
-
when line.
|
44
|
+
when line.properties['block_name']
|
45
|
+
# unnamed blocks are not run
|
43
46
|
@current_tangle = nil
|
44
47
|
@buffer = ''
|
45
48
|
|
@@ -50,11 +53,12 @@ module Orgmode
|
|
50
53
|
:name => line.properties['block_name']
|
51
54
|
}
|
52
55
|
@scripts[@scripts_counter][:lang] = line.block_lang
|
53
|
-
|
54
56
|
# TODO: have a way to specify which are the default binaries to be used per language
|
55
57
|
# when binary_detected?(@block_lang)
|
56
58
|
else
|
57
|
-
#
|
59
|
+
# reset tangling
|
60
|
+
@current_tangle = nil
|
61
|
+
@buffer = ''
|
58
62
|
end
|
59
63
|
end
|
60
64
|
|
@@ -66,7 +70,8 @@ module Orgmode
|
|
66
70
|
# When a tangle is not going on, it means that the lines would go
|
67
71
|
# into a runnable script
|
68
72
|
@buffer << line.output_text << "\n"
|
69
|
-
when (!@buffer.empty? and not (line.begin_block? or
|
73
|
+
when (!@buffer.empty? and not (line.begin_block? or \
|
74
|
+
line.assigned_paragraph_type == :code))
|
70
75
|
# Fix indentation and remove fix commas from Org mode before flushing
|
71
76
|
strip_code_block!
|
72
77
|
@scripts[@scripts_counter][:lines] << @buffer
|
data/lib/org-converge/command.rb
CHANGED
@@ -37,14 +37,24 @@ module OrgConverge
|
|
37
37
|
|
38
38
|
def converge!
|
39
39
|
tangle!
|
40
|
-
|
40
|
+
if @options['--runmode']
|
41
|
+
dispatch_runmode(@options['--runmode'])
|
42
|
+
else
|
43
|
+
# Try to find one in the buffer
|
44
|
+
runmode = ob.in_buffer_settings['RUNMODE']
|
45
|
+
dispatch_runmode(runmode)
|
46
|
+
end
|
47
|
+
end
|
48
|
+
|
49
|
+
def dispatch_runmode(runmode)
|
50
|
+
case runmode
|
41
51
|
when 'parallel'
|
42
52
|
run_blocks_in_parallel!
|
43
53
|
when 'sequential'
|
44
54
|
run_blocks_sequentially!
|
45
55
|
when 'runlist'
|
46
56
|
# TODO
|
47
|
-
else
|
57
|
+
else # parallel by default
|
48
58
|
run_blocks_in_parallel!
|
49
59
|
end
|
50
60
|
end
|
@@ -52,7 +62,7 @@ module OrgConverge
|
|
52
62
|
def tangle!
|
53
63
|
results = babel.tangle!
|
54
64
|
rescue Orgmode::Babel::TangleError
|
55
|
-
logger.error "Cannot converge because there were errors during tangle step".red
|
65
|
+
logger.error "Cannot converge because there were errors during tangle step".fg 'red'
|
56
66
|
end
|
57
67
|
|
58
68
|
def run_blocks_sequentially!
|
@@ -66,14 +76,18 @@ module OrgConverge
|
|
66
76
|
|
67
77
|
while not runlist_stack.empty?
|
68
78
|
key, script = runlist_stack.shift
|
79
|
+
|
80
|
+
# Decision: Only run blocks which have a name
|
81
|
+
next unless script[:header][:name]
|
82
|
+
|
83
|
+
display_name = script[:header][:name]
|
69
84
|
with_running_engine do |engine|
|
70
85
|
file = File.expand_path("#{@run_dir}/#{key}")
|
71
86
|
cmd = "#{script[:lang]} #{file}"
|
72
|
-
display_name = script[:header][:name] || script[:lang]
|
73
87
|
engine.register display_name, cmd, { :cwd => @root_dir, :logger => logger }
|
74
88
|
end
|
75
89
|
end
|
76
|
-
logger.info "Run has completed successfully.".green
|
90
|
+
logger.info "Run has completed successfully.".fg 'green'
|
77
91
|
end
|
78
92
|
|
79
93
|
# TODO: Too much foreman has made this running blocks in parallel the default behavior.
|
@@ -84,12 +98,16 @@ module OrgConverge
|
|
84
98
|
babel.ob.scripts.each do |key, script|
|
85
99
|
file = File.expand_path("#{@run_dir}/#{key}")
|
86
100
|
cmd = "#{script[:lang]} #{file}"
|
87
|
-
|
101
|
+
|
102
|
+
# Decision: Only run blocks which have a name
|
103
|
+
next unless script[:header][:name]
|
104
|
+
|
105
|
+
display_name = script[:header][:name]
|
88
106
|
@engine.register display_name, cmd, { :cwd => @root_dir, :logger => logger }
|
89
107
|
end
|
90
108
|
logger.info "Running code blocks now! (#{babel.ob.scripts.count} runnable blocks found in total)"
|
91
109
|
@engine.start
|
92
|
-
logger.info "Run has completed successfully.".green
|
110
|
+
logger.info "Run has completed successfully.".fg 'green'
|
93
111
|
end
|
94
112
|
|
95
113
|
def with_running_engine
|
@@ -104,12 +122,12 @@ module OrgConverge
|
|
104
122
|
|
105
123
|
def showfiles
|
106
124
|
ob.tangle.each do |file, block|
|
107
|
-
puts "---------- #{file} --------------".green
|
125
|
+
puts "---------- #{file} --------------".fg 'green'
|
108
126
|
puts block[:lines]
|
109
127
|
end
|
110
128
|
|
111
129
|
ob.scripts.each do |index, block|
|
112
|
-
puts "---------- script: #{index} to be run with: #{block[:header][:shebang]} --------------".green
|
130
|
+
puts "---------- script: #{index} to be run with: #{block[:header][:shebang]} --------------".fg 'green'
|
113
131
|
puts block[:lines]
|
114
132
|
end
|
115
133
|
end
|
data/lib/org-converge/version.rb
CHANGED
data/lib/org-converge.rb
CHANGED
@@ -14,7 +14,10 @@ module Orgmode
|
|
14
14
|
# Feed the parsed contens and create the necessary internal structures
|
15
15
|
# for doing babel like features
|
16
16
|
output = ''
|
17
|
-
|
17
|
+
babel_options = {
|
18
|
+
:in_buffer_settings => @in_buffer_settings
|
19
|
+
}
|
20
|
+
ob = BabelOutputBuffer.new(output, babel_options)
|
18
21
|
translate(@header_lines, ob)
|
19
22
|
@headlines.each do |headline|
|
20
23
|
translate(headline.body_lines, ob)
|
@@ -25,20 +28,8 @@ module Orgmode
|
|
25
28
|
end
|
26
29
|
end
|
27
30
|
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
def magenta; colorize("\e[0m\e[35m"); end
|
34
|
-
def cyan; colorize("\e[0m\e[36m"); end
|
35
|
-
def white; colorize("\e[0m\e[37m"); end
|
36
|
-
def bright_black; colorize("\e[0m\e[30m"); end
|
37
|
-
def bright_magenta; colorize("\e[0m\e[35m"); end
|
38
|
-
def bold; colorize("\e[0m\e[1m"); end
|
39
|
-
def colorize(color_code); "#{color_code}#{self}\e[0m"; end
|
40
|
-
end
|
41
|
-
|
42
|
-
class String
|
43
|
-
include StringWithColors
|
44
|
-
end
|
31
|
+
require 'tco'
|
32
|
+
conf = Tco.config
|
33
|
+
conf.names["green"] = "#02a552"
|
34
|
+
conf.names["red"] = "#eb443b"
|
35
|
+
Tco.reconfigure conf
|
data/org-converge.gemspec
CHANGED
@@ -15,7 +15,7 @@ Gem::Specification.new do |gem|
|
|
15
15
|
gem.require_paths = ["lib"]
|
16
16
|
gem.version = OrgConverge::VERSION
|
17
17
|
gem.add_runtime_dependency('docopt', '0.5.0')
|
18
|
-
gem.add_runtime_dependency('org-ruby', '~> 0.9.
|
18
|
+
gem.add_runtime_dependency('org-ruby', '~> 0.9.4')
|
19
19
|
gem.add_runtime_dependency('foreman', '~> 0.63.0')
|
20
20
|
gem.add_runtime_dependency('tco', '~> 0.1.0')
|
21
21
|
end
|
@@ -1,33 +1,45 @@
|
|
1
|
-
#+
|
1
|
+
#+TITLE: Running scripts in parallel
|
2
2
|
|
3
3
|
In the following example, the following 3 scripts should be run in parallel
|
4
4
|
with all output being flushed to the screen.
|
5
5
|
|
6
6
|
- Count some numbers with bash
|
7
7
|
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
8
|
+
#+name: bash_counter
|
9
|
+
#+begin_src sh :shebang #!/bin/bash
|
10
|
+
echo '' > out.log
|
11
|
+
for i in `seq 1 5`; do
|
12
|
+
echo "Writing! $i"
|
13
|
+
echo "hello $i" >> out.log
|
14
|
+
sleep $(($RANDOM % 5))
|
15
|
+
done
|
16
|
+
#+end_src
|
16
17
|
|
17
18
|
- Count some numbers with ruby
|
18
19
|
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
20
|
+
#+name: ruby_counter
|
21
|
+
#+begin_src ruby :shebang #!/usr/bin/ruby
|
22
|
+
$stdout.sync = true
|
23
|
+
sleep 0.2
|
24
|
+
10.times do |n|
|
25
|
+
puts "And now writing! #{n}"
|
26
|
+
File.open("out.log", "a") {|f| f.puts "Hello again #{n}" }
|
27
|
+
sleep rand.round(2)
|
28
|
+
end
|
29
|
+
#+end_src
|
27
30
|
|
28
31
|
- Print some numbers with python
|
29
32
|
|
33
|
+
#+name: python_counter
|
30
34
|
#+begin_src python :shebang #!/usr/bin/python
|
31
35
|
for i in range(0,3):
|
32
36
|
print i
|
33
37
|
#+end_src
|
38
|
+
|
39
|
+
- Block that should not be run
|
40
|
+
|
41
|
+
#+begin_src js
|
42
|
+
{
|
43
|
+
"hello": "world"
|
44
|
+
}
|
45
|
+
#+end_src
|
@@ -1,8 +1,10 @@
|
|
1
1
|
# -*- mode: org; mode: auto-fill; -*-
|
2
|
-
#+TITLE:
|
2
|
+
#+TITLE: Runlist example
|
3
|
+
#+runmode: sequential
|
3
4
|
|
4
5
|
In this example, we will have scripts being chained one after another.
|
5
6
|
|
7
|
+
#+name: first
|
6
8
|
#+begin_src sh :shebang #!/bin/bash
|
7
9
|
sleep 5
|
8
10
|
echo "first" > out.log
|
@@ -10,6 +12,7 @@ echo "first" > out.log
|
|
10
12
|
|
11
13
|
The script below will only be called once the one above is run.
|
12
14
|
|
15
|
+
#+name: second
|
13
16
|
#+begin_src ruby :shebang #!/usr/local/bin/ruby
|
14
17
|
File.open("out.log", 'a') {|f| f.puts "second" }
|
15
18
|
#+end_src
|
data/spec/converge_spec.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: org-converge
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.5
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -9,11 +9,11 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2014-04-
|
12
|
+
date: 2014-04-20 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: docopt
|
16
|
-
requirement: &
|
16
|
+
requirement: &70344197198100 !ruby/object:Gem::Requirement
|
17
17
|
none: false
|
18
18
|
requirements:
|
19
19
|
- - =
|
@@ -21,21 +21,21 @@ dependencies:
|
|
21
21
|
version: 0.5.0
|
22
22
|
type: :runtime
|
23
23
|
prerelease: false
|
24
|
-
version_requirements: *
|
24
|
+
version_requirements: *70344197198100
|
25
25
|
- !ruby/object:Gem::Dependency
|
26
26
|
name: org-ruby
|
27
|
-
requirement: &
|
27
|
+
requirement: &70344197197620 !ruby/object:Gem::Requirement
|
28
28
|
none: false
|
29
29
|
requirements:
|
30
30
|
- - ~>
|
31
31
|
- !ruby/object:Gem::Version
|
32
|
-
version: 0.9.
|
32
|
+
version: 0.9.4
|
33
33
|
type: :runtime
|
34
34
|
prerelease: false
|
35
|
-
version_requirements: *
|
35
|
+
version_requirements: *70344197197620
|
36
36
|
- !ruby/object:Gem::Dependency
|
37
37
|
name: foreman
|
38
|
-
requirement: &
|
38
|
+
requirement: &70344196892060 !ruby/object:Gem::Requirement
|
39
39
|
none: false
|
40
40
|
requirements:
|
41
41
|
- - ~>
|
@@ -43,10 +43,10 @@ dependencies:
|
|
43
43
|
version: 0.63.0
|
44
44
|
type: :runtime
|
45
45
|
prerelease: false
|
46
|
-
version_requirements: *
|
46
|
+
version_requirements: *70344196892060
|
47
47
|
- !ruby/object:Gem::Dependency
|
48
48
|
name: tco
|
49
|
-
requirement: &
|
49
|
+
requirement: &70344196890840 !ruby/object:Gem::Requirement
|
50
50
|
none: false
|
51
51
|
requirements:
|
52
52
|
- - ~>
|
@@ -54,7 +54,7 @@ dependencies:
|
|
54
54
|
version: 0.1.0
|
55
55
|
type: :runtime
|
56
56
|
prerelease: false
|
57
|
-
version_requirements: *
|
57
|
+
version_requirements: *70344196890840
|
58
58
|
description: A light configuration management tool for Org mode
|
59
59
|
email:
|
60
60
|
- waldemar.quevedo@gmail.com
|
@@ -67,7 +67,6 @@ files:
|
|
67
67
|
- .travis.yml
|
68
68
|
- Gemfile
|
69
69
|
- LICENSE
|
70
|
-
- README.md
|
71
70
|
- README.org
|
72
71
|
- Rakefile
|
73
72
|
- TODO
|
data/README.md
DELETED
@@ -1,157 +0,0 @@
|
|
1
|
-
Org Converge
|
2
|
-
------------
|
3
|
-
[![Build Status](https://travis-ci.org/wallyqs/org-converge.svg?branch=master)](https://travis-ci.org/wallyqs/org-converge)
|
4
|
-
|
5
|
-
# Description
|
6
|
-
|
7
|
-
A tool which uses Org mode syntax to describe, configure and
|
8
|
-
setup something, borrowing some ideas of what is possible to
|
9
|
-
do with tools like `chef-solo`, `puppet`, `ansible`, etc...
|
10
|
-
|
11
|
-
Can also be used for tangling code blocks like Org Babel does.
|
12
|
-
|
13
|
-
# Installing
|
14
|
-
|
15
|
-
$ gem install org-converge
|
16
|
-
|
17
|
-
# Motivation
|
18
|
-
|
19
|
-
The Org babel syntax has proven to be flexible enough to produce
|
20
|
-
*reproducible research* papers. Then, I believe that configuring and setting up
|
21
|
-
a server for example is something that could be also be done using
|
22
|
-
the same syntax, given that *converging* the configuration is something
|
23
|
-
that one ought to be able to reproduce.
|
24
|
-
|
25
|
-
# Example usage
|
26
|
-
|
27
|
-
Inspecting the files that would be tangled:
|
28
|
-
|
29
|
-
$ org-converge spec/converge_examples/basic_tangle/setup.org --showfiles
|
30
|
-
|
31
|
-
---------- conf.yml --------------
|
32
|
-
bind: 0.0.0.0
|
33
|
-
port: 2442
|
34
|
-
|
35
|
-
mysql:
|
36
|
-
db: users
|
37
|
-
host: somewhere-example.local
|
38
|
-
password: 111111111
|
39
|
-
|
40
|
-
Executing the tangle:
|
41
|
-
|
42
|
-
$ org-converge spec/converge_examples/basic_tangle/setup.org --tangle
|
43
|
-
|
44
|
-
I, [2014-03-24T00:21:08.073506 #660] INFO -- : Tangling 1 files...
|
45
|
-
I, [2014-03-24T00:21:08.073668 #660] INFO -- : BEGIN(conf.yml): Tangling 7 lines at 'conf.yml'
|
46
|
-
I, [2014-03-24T00:21:08.075562 #660] INFO -- : END(conf.yml): done.
|
47
|
-
I, [2014-03-24T00:21:08.075638 #660] INFO -- : Tangling succeeded!
|
48
|
-
|
49
|
-
Run the code blocks in sequence:
|
50
|
-
|
51
|
-
$ org-converge spec/converge_examples/runlist_example/setup.org --runmode=sequential
|
52
|
-
I, [2014-04-02T01:18:17.336255 #65462] INFO -- : Tangling 0 files...
|
53
|
-
I, [2014-04-02T01:18:17.336376 #65462] INFO -- : Tangling succeeded!
|
54
|
-
I, [2014-04-02T01:18:17.336698 #65462] INFO -- : Tangling 2 scripts within directory: /Users/mariko/repos/org-converge/run...
|
55
|
-
I, [2014-04-02T01:18:17.340638 #65462] INFO -- : sh(65466) -- started with pid 65466
|
56
|
-
I, [2014-04-02T01:18:22.365128 #65462] INFO -- : sh(65466) -- exited with code 0
|
57
|
-
I, [2014-04-02T01:18:22.398983 #65462] INFO -- : ruby(65469) -- started with pid 65469
|
58
|
-
I, [2014-04-02T01:18:23.013195 #65462] INFO -- : ruby(65469) -- exited with code 0
|
59
|
-
I, [2014-04-02T01:18:23.013354 #65462] INFO -- : Run has completed successfully.
|
60
|
-
|
61
|
-
Running the code blocks in parallel mode is also possible:
|
62
|
-
|
63
|
-
$ org-converge spec/converge_examples/basic_run_example/setup.org --runmode=parallel
|
64
|
-
|
65
|
-
I, [2014-04-02T01:16:27.660259 #65126] INFO -- : Tangling 0 files...
|
66
|
-
I, [2014-04-02T01:16:27.660390 #65126] INFO -- : Tangling succeeded!
|
67
|
-
I, [2014-04-02T01:16:27.661640 #65126] INFO -- : Running code blocks now! (3 runnable blocks found in total)
|
68
|
-
I, [2014-04-02T01:16:27.732659 #65126] INFO -- : sh(65157) -- started with pid 65157
|
69
|
-
I, [2014-04-02T01:16:27.732819 #65126] INFO -- : ruby(65158) -- started with pid 65158
|
70
|
-
I, [2014-04-02T01:16:27.732915 #65126] INFO -- : python(65160) -- started with pid 65160
|
71
|
-
I, [2014-04-02T01:16:27.733565 #65126] INFO -- : sh(65157) -- Writing! 1
|
72
|
-
I, [2014-04-02T01:16:27.771819 #65126] INFO -- : sh(65157) -- Writing! 2
|
73
|
-
I, [2014-04-02T01:16:27.914407 #65126] INFO -- : python(65160) -- 0
|
74
|
-
I, [2014-04-02T01:16:27.914674 #65126] INFO -- : python(65160) -- 1
|
75
|
-
I, [2014-04-02T01:16:27.914887 #65126] INFO -- : python(65160) -- 2
|
76
|
-
I, [2014-04-02T01:16:27.921333 #65126] INFO -- : python(65160) -- exited with code 0
|
77
|
-
I, [2014-04-02T01:16:33.226998 #65126] INFO -- : ruby(65158) -- And now writing! 0
|
78
|
-
I, [2014-04-02T01:16:33.227257 #65126] INFO -- : ruby(65158) -- And now writing! 1
|
79
|
-
I, [2014-04-02T01:16:33.227458 #65126] INFO -- : ruby(65158) -- And now writing! 2
|
80
|
-
I, [2014-04-02T01:16:33.227673 #65126] INFO -- : ruby(65158) -- And now writing! 3
|
81
|
-
I, [2014-04-02T01:16:33.248160 #65126] INFO -- : ruby(65158) -- And now writing! 4
|
82
|
-
I, [2014-04-02T01:16:33.248378 #65126] INFO -- : ruby(65158) -- And now writing! 5
|
83
|
-
I, [2014-04-02T01:16:33.248552 #65126] INFO -- : ruby(65158) -- And now writing! 6
|
84
|
-
I, [2014-04-02T01:16:33.301512 #65126] INFO -- : ruby(65158) -- And now writing! 7
|
85
|
-
I, [2014-04-02T01:16:33.301877 #65126] INFO -- : ruby(65158) -- And now writing! 8
|
86
|
-
I, [2014-04-02T01:16:33.302186 #65126] INFO -- : ruby(65158) -- And now writing! 9
|
87
|
-
I, [2014-04-02T01:16:33.363135 #65126] INFO -- : ruby(65158) -- exited with code 0
|
88
|
-
I, [2014-04-02T01:16:36.736210 #65126] INFO -- : sh(65157) -- Writing! 3
|
89
|
-
I, [2014-04-02T01:16:41.747375 #65126] INFO -- : sh(65157) -- Writing! 4
|
90
|
-
I, [2014-04-02T01:16:47.754810 #65126] INFO -- : sh(65157) -- Writing! 5
|
91
|
-
I, [2014-04-02T01:16:50.764513 #65126] INFO -- : sh(65157) -- exited with code 0
|
92
|
-
I, [2014-04-02T01:16:50.764636 #65126] INFO -- : Run has completed successfully.
|
93
|
-
|
94
|
-
# How it works
|
95
|
-
|
96
|
-
Org Converge uses an liberally extended version of Org Babel
|
97
|
-
features in order to give support for converging the configuration
|
98
|
-
of a server.
|
99
|
-
|
100
|
-
For example, using Org Babel and macros we can easily spread config
|
101
|
-
files on a server by writing the following on a `server.org` file.
|
102
|
-
|
103
|
-
```org
|
104
|
-
#+begin_src yaml :tangle /etc/component.yml
|
105
|
-
multitenant: false
|
106
|
-
status_port: 10004
|
107
|
-
#+end_src
|
108
|
-
```
|
109
|
-
|
110
|
-
And then configure it by running it as follows, (considering we have
|
111
|
-
the correct permissions):
|
112
|
-
|
113
|
-
```sh
|
114
|
-
$ org-converge server.org
|
115
|
-
```
|
116
|
-
|
117
|
-
Next, let's say that we no only one want to set the configured templates,
|
118
|
-
but that we also want to install some packages. In that case, we
|
119
|
-
should be able to do the following:
|
120
|
-
|
121
|
-
```org
|
122
|
-
* Configuring the component
|
123
|
-
|
124
|
-
#+begin_src yaml :tangle /etc/component.yml
|
125
|
-
multitenant: false
|
126
|
-
status_port: 10004
|
127
|
-
#+end_src
|
128
|
-
|
129
|
-
* Installing the dependencies
|
130
|
-
|
131
|
-
Need the following so that ~bundle install~ can compile
|
132
|
-
the native extensions correctly.
|
133
|
-
|
134
|
-
#+begin_src sh :shebang #!/bin/bash
|
135
|
-
apt-get install build-essentials -y
|
136
|
-
#+end_src
|
137
|
-
|
138
|
-
Then the following should work:
|
139
|
-
|
140
|
-
#+begin_src sh :shebang #!/bin/bash
|
141
|
-
cd {{{project_path}}}
|
142
|
-
bundle install
|
143
|
-
#+end_src
|
144
|
-
```
|
145
|
-
|
146
|
-
As long as the repo has been already checked out in the directory,
|
147
|
-
the previous example will succeed. Note that we are also setting
|
148
|
-
the `:shebang` on the scripts so that the script is executable.
|
149
|
-
|
150
|
-
More practical examples can be found in the examples directory.
|
151
|
-
Many more will be added as long as dogfooding from this goes well.
|
152
|
-
|
153
|
-
# Contributing
|
154
|
-
|
155
|
-
The project is in very early development at this moment, but if you
|
156
|
-
feel that it is interesting enough, please create a ticket to start
|
157
|
-
the discussion.
|