org-converge 0.0.7 → 0.0.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.org +91 -86
- data/TODO +3 -3
- data/bin/org-spec +29 -0
- data/lib/org-converge/babel_output_buffer.rb +13 -5
- data/lib/org-converge/command.rb +152 -15
- data/lib/org-converge/engine.rb +28 -7
- data/lib/org-converge/version.rb +1 -1
- data/org-converge.gemspec +1 -1
- data/spec/converge_examples/expected_results/spec.org +62 -0
- data/spec/converge_examples/expected_results/spec2.org +62 -0
- data/spec/converge_spec.rb +26 -0
- metadata +19 -13
data/README.org
CHANGED
@@ -2,45 +2,64 @@
|
|
2
2
|
#+STARTUP: showeverything
|
3
3
|
|
4
4
|
* Org Converge
|
5
|
+
|
6
|
+
[[https://secure.travis-ci.org/wallyqs/org-converge.png?branch=master]]
|
7
|
+
|
5
8
|
** Description
|
6
9
|
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
10
|
+
This attempts to be an experiment of using Org mode syntax to
|
11
|
+
create documentable reproducible runs, borrowing some ideas
|
12
|
+
of what is possible to do with tools like =chef-solo=,
|
13
|
+
=rake=, =foreman=, etc...
|
14
|
+
|
15
|
+
** Installing
|
16
|
+
|
17
|
+
: gem install org-converge
|
11
18
|
|
12
19
|
** Motivation
|
13
20
|
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
21
|
+
The Org babel syntax has proven to be flexible enough to writing
|
22
|
+
/reproducible research/ papers. Then, I believe that configuring and setting up
|
23
|
+
a server for example is something that could be also be done using
|
24
|
+
the same syntax, given that /converging/ the configuration is something
|
25
|
+
that one ought to be able to reproduce.
|
19
26
|
|
20
27
|
** Usage
|
21
28
|
|
22
|
-
|
29
|
+
To run the blocks in parallel...
|
23
30
|
|
24
31
|
#+begin_src sh
|
25
|
-
org-converge path/to/setup-file.org
|
32
|
+
org-converge path/to/setup-file.org --runmode=parallel
|
26
33
|
#+end_src
|
27
34
|
|
28
|
-
|
35
|
+
or sequentially...
|
29
36
|
|
30
37
|
#+begin_src sh
|
31
38
|
org-converge path/to/setup-file.org --runmode=sequential
|
32
39
|
#+end_src
|
33
40
|
|
41
|
+
By including ~:after~ or ~:before~ arguments to a block,
|
42
|
+
it is possible to make the blocks depend on one another.
|
43
|
+
|
44
|
+
#+begin_src sh
|
45
|
+
org-converge path/to/setup-file.org --runmode=chained
|
46
|
+
#+end_src
|
47
|
+
|
48
|
+
*** Other commands available
|
49
|
+
|
50
|
+
: org-run # alias for org-converge
|
51
|
+
: org-tangle # just tangles the contents without running the blocks
|
52
|
+
|
34
53
|
** Quick example
|
35
54
|
|
36
|
-
|
37
|
-
|
38
|
-
|
55
|
+
The following is an example of running 3 processes
|
56
|
+
in parallel by defining them as code blocks from
|
57
|
+
an Org mode file:
|
39
58
|
|
40
59
|
#+begin_src sh
|
41
60
|
,#+TITLE: Running Org babel processes in parallel
|
42
61
|
|
43
|
-
|
62
|
+
,* Print with different languages
|
44
63
|
|
45
64
|
,#+name: hello_from_bash
|
46
65
|
,#+begin_src sh :shebang #!/bin/bash
|
@@ -64,96 +83,82 @@ org-converge path/to/setup-file.org --runmode=sequential
|
|
64
83
|
,#+end_src
|
65
84
|
#+end_src
|
66
85
|
|
67
|
-
|
86
|
+
We store this in a file named =hello.org= and then run it as follows:
|
68
87
|
|
69
88
|
#+begin_src sh
|
70
89
|
org-converge hello.org
|
71
90
|
#+end_src
|
72
91
|
|
73
|
-
|
92
|
+
This would produce an output similar to the following:
|
74
93
|
|
75
94
|
#+begin_src sh
|
76
|
-
|
77
|
-
|
78
|
-
|
79
|
-
|
80
|
-
|
81
|
-
|
82
|
-
|
83
|
-
|
84
|
-
|
85
|
-
|
86
|
-
|
87
|
-
I, [2014-04-17T23:48:45.792321 #1710] INFO -- : hello_from_ruby (1712) -- hello world from ruby
|
88
|
-
I, [2014-04-17T23:48:45.882020 #1710] INFO -- : hello_from_python (1713) -- hello world from python
|
89
|
-
I, [2014-04-17T23:48:46.787106 #1710] INFO -- : hello_from_bash (1711) -- hello world from bash
|
90
|
-
I, [2014-04-17T23:48:46.793347 #1710] INFO -- : hello_from_ruby (1712) -- hello world from ruby
|
91
|
-
I, [2014-04-17T23:48:46.883511 #1710] INFO -- : hello_from_python (1713) -- hello world from python
|
92
|
-
I, [2014-04-17T23:48:47.794846 #1710] INFO -- : hello_from_ruby (1712) -- hello world from ruby
|
93
|
-
I, [2014-04-17T23:48:47.796387 #1710] INFO -- : hello_from_bash (1711) -- hello world from bash
|
94
|
-
I, [2014-04-17T23:48:47.884196 #1710] INFO -- : hello_from_python (1713) -- hello world from python
|
95
|
+
[2014-05-04T19:23:40 +0900] Tangling 0 files...
|
96
|
+
[2014-05-04T19:23:40 +0900] Tangling succeeded!
|
97
|
+
[2014-05-04T19:23:40 +0900] Tangling 3 scripts within directory: /Users/mariko/repos/org-converge/run...
|
98
|
+
[2014-05-04T19:23:40 +0900] Running code blocks now! (3 runnable blocks found in total)
|
99
|
+
[2014-05-04T19:23:40 +0900] hello_from_bash (4664) -- started with pid 4664
|
100
|
+
[2014-05-04T19:23:40 +0900] hello_from_ruby (4665) -- started with pid 4665
|
101
|
+
[2014-05-04T19:23:40 +0900] hello_from_python (4666) -- started with pid 4666
|
102
|
+
[2014-05-04T19:23:40 +0900] hello_from_bash (4664) -- hello world from bash
|
103
|
+
[2014-05-04T19:23:41 +0900] hello_from_ruby (4665) -- hello world from ruby
|
104
|
+
[2014-05-04T19:23:41 +0900] hello_from_python (4666) -- hello world from python
|
105
|
+
[2014-05-04T19:23:42 +0900] hello_from_ruby (4665) -- hello world from ruby
|
95
106
|
#+end_src
|
96
107
|
|
97
108
|
** How it works
|
98
109
|
|
99
|
-
|
100
|
-
|
101
|
-
|
110
|
+
Org Converge uses an liberally extended version of Org Babel
|
111
|
+
features in order to give support for converging the configuration
|
112
|
+
of a server.
|
113
|
+
|
114
|
+
For example, using Org Babel and macros we can easily spread config
|
115
|
+
files on a server by writing the following on a ~server.org~ file.
|
116
|
+
|
117
|
+
#+begin_src sh
|
118
|
+
,#+begin_src yaml :tangle /etc/component.yml
|
119
|
+
multitenant: false
|
120
|
+
status_port: 10004
|
121
|
+
,#+end_src
|
122
|
+
#+end_src
|
123
|
+
|
124
|
+
And then configure it by running it as follows, (considering we have
|
125
|
+
the correct permissions for tangling at =/etc/component.yml=):
|
102
126
|
|
103
|
-
|
104
|
-
|
127
|
+
#+begin_src sh
|
128
|
+
sudo org-converge server.org
|
129
|
+
#+end_src
|
105
130
|
|
106
|
-
|
107
|
-
|
108
|
-
|
109
|
-
status_port: 10004
|
110
|
-
,#+end_src
|
111
|
-
#+end_src
|
131
|
+
Next, let's say that we no only one want to set the configured templates,
|
132
|
+
but that we also want to install some packages. In that case, we
|
133
|
+
should be able to do the following:
|
112
134
|
|
113
|
-
|
114
|
-
|
135
|
+
#+begin_src sh
|
136
|
+
,* Configuring the component
|
115
137
|
|
116
|
-
|
117
|
-
|
118
|
-
|
138
|
+
,#+begin_src yaml :tangle /etc/component.yml
|
139
|
+
multitenant: false
|
140
|
+
status_port: 10004
|
141
|
+
,#+end_src
|
119
142
|
|
120
|
-
|
121
|
-
but that we also want to install some packages. In that case, we
|
122
|
-
should be able to do the following:
|
143
|
+
,* Installing the dependencies
|
123
144
|
|
124
|
-
|
125
|
-
|
145
|
+
Need the following so that ~bundle install~ can compile
|
146
|
+
the native extensions correctly.
|
126
147
|
|
127
|
-
|
128
|
-
|
129
|
-
|
130
|
-
,#+end_src
|
131
|
-
|
132
|
-
,* Installing the dependencies
|
148
|
+
,#+begin_src sh
|
149
|
+
apt-get install build-essentials -y
|
150
|
+
,#+end_src
|
133
151
|
|
134
|
-
|
135
|
-
the native extensions correctly.
|
152
|
+
Then the following should work:
|
136
153
|
|
137
|
-
|
138
|
-
|
139
|
-
|
140
|
-
|
141
|
-
|
142
|
-
|
143
|
-
,#+begin_src sh
|
144
|
-
cd project_path
|
145
|
-
bundle install
|
146
|
-
,#+end_src
|
147
|
-
#+end_src
|
148
|
-
|
149
|
-
As long as the repo has been already checked out in the directory,
|
150
|
-
the previous example will succeed.
|
151
|
-
|
152
|
-
More practical examples can be found [[https://github.com/wallyqs/org-converge/tree/master/examples][here]], more will be added as
|
153
|
-
long as dogfooding from this goes well.
|
154
|
+
,#+begin_src sh
|
155
|
+
cd project_path
|
156
|
+
bundle install
|
157
|
+
,#+end_src
|
158
|
+
#+end_src
|
154
159
|
|
155
160
|
** Contributing
|
156
161
|
|
157
|
-
|
158
|
-
|
159
|
-
|
162
|
+
The project is in very early development at this moment, but if you
|
163
|
+
feel that it is interesting enough, please create a ticket to start
|
164
|
+
the discussion.
|
data/TODO
CHANGED
@@ -5,7 +5,7 @@
|
|
5
5
|
* [0/7] > 0.0.9 version
|
6
6
|
|
7
7
|
- [ ] Macros can be loaded and applied to the configuration
|
8
|
-
- [ ] Actually support converging and idempotency
|
8
|
+
- [ ] Actually support converging and idempotency (~--runmode=idempotent~)
|
9
9
|
+ Do not do an operation unless it is required
|
10
10
|
+ Abort in case there was a failure in executing the script.
|
11
11
|
+ This would work by using the ~:cache true~ to a block header argument
|
@@ -16,9 +16,9 @@
|
|
16
16
|
- [ ] Use :eval for evaling blocks (off by default)
|
17
17
|
- [ ] Can use :dir for running a process relative to that directory
|
18
18
|
|
19
|
-
* [
|
19
|
+
* [1/1] 0.0.8 version
|
20
20
|
|
21
|
-
- [
|
21
|
+
- [X] ~org-spec~ exploration!
|
22
22
|
|
23
23
|
* [2/2] 0.0.7 version
|
24
24
|
|
data/bin/org-spec
ADDED
@@ -0,0 +1,29 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
# -*- mode: ruby -*-
|
3
|
+
require 'docopt'
|
4
|
+
require 'org-converge'
|
5
|
+
|
6
|
+
doc = <<OPTIONS
|
7
|
+
|
8
|
+
org-spec: Validate whether code blocks match to the contents of the #+RESULTS block
|
9
|
+
|
10
|
+
Usage:
|
11
|
+
org-spec <org_file> [--showfiles] [--log=<logfile>] [--root-dir=<root_dir>] [--name=<block_name>]
|
12
|
+
|
13
|
+
Options:
|
14
|
+
|
15
|
+
-h --help Show this screen.
|
16
|
+
|
17
|
+
OPTIONS
|
18
|
+
|
19
|
+
begin
|
20
|
+
require "pp"
|
21
|
+
cmd = Docopt::docopt(doc)
|
22
|
+
rescue Docopt::Exit => e
|
23
|
+
puts e.message
|
24
|
+
end
|
25
|
+
|
26
|
+
exit 1 unless cmd
|
27
|
+
|
28
|
+
o = OrgConverge::Command.new(cmd.merge({ '--runmode' => 'spec' }))
|
29
|
+
o.execute!
|
@@ -23,9 +23,9 @@ module Orgmode
|
|
23
23
|
# ~@scripts~ are tangled in order and ran
|
24
24
|
# : @scripts = [text, text, ...]
|
25
25
|
@scripts = Hash.new {|h,k| h[k] = {
|
26
|
-
:lines
|
27
|
-
:header
|
28
|
-
:lang
|
26
|
+
:lines => '',
|
27
|
+
:header => {},
|
28
|
+
:lang => ''
|
29
29
|
}
|
30
30
|
}
|
31
31
|
@scripts_counter = 0
|
@@ -77,10 +77,13 @@ module Orgmode
|
|
77
77
|
end
|
78
78
|
|
79
79
|
case
|
80
|
+
when (line.start_of_results_code_block?)
|
81
|
+
@accumulate_results_block = true
|
80
82
|
when (line.assigned_paragraph_type == :code and @current_tangle)
|
81
83
|
# Need to keep track of the current tangle to buffer its lines
|
82
84
|
@tangle[@current_tangle][:lines] << line.output_text << "\n"
|
83
|
-
when (line.assigned_paragraph_type == :code
|
85
|
+
when (line.assigned_paragraph_type == :code or \
|
86
|
+
line.paragraph_type == :inline_example)
|
84
87
|
# When a tangle is not going on, it means that the lines would go
|
85
88
|
# into a runnable script
|
86
89
|
@buffer << line.output_text << "\n"
|
@@ -88,7 +91,12 @@ module Orgmode
|
|
88
91
|
line.assigned_paragraph_type == :code))
|
89
92
|
# Fix indentation and remove fix commas from Org mode before flushing
|
90
93
|
strip_code_block!
|
91
|
-
|
94
|
+
if @accumulate_results_block
|
95
|
+
@scripts[@scripts_counter - 1][:results] = @buffer
|
96
|
+
@accumulate_results_block = false
|
97
|
+
else
|
98
|
+
@scripts[@scripts_counter][:lines] << @buffer
|
99
|
+
end
|
92
100
|
@buffer = ''
|
93
101
|
@scripts_counter += 1
|
94
102
|
end
|
data/lib/org-converge/command.rb
CHANGED
@@ -6,14 +6,21 @@ module OrgConverge
|
|
6
6
|
attr_reader :engine
|
7
7
|
|
8
8
|
def initialize(options)
|
9
|
-
@options
|
10
|
-
@dotorg
|
11
|
-
@root_dir
|
12
|
-
@run_dir
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
9
|
+
@options = options
|
10
|
+
@dotorg = options['<org_file>']
|
11
|
+
@root_dir = options['--root-dir']
|
12
|
+
@run_dir = if @root_dir
|
13
|
+
File.expand_path(File.join(@root_dir, 'run'))
|
14
|
+
else
|
15
|
+
File.expand_path('run')
|
16
|
+
end
|
17
|
+
# The results dir will have a timestamp to avoid having to refresh all the time
|
18
|
+
results_dirname = "results_#{Time.now.strftime("%Y%m%d%H%M%S")}"
|
19
|
+
@results_dir = if @root_dir
|
20
|
+
File.expand_path(File.join(@root_dir, results_dirname))
|
21
|
+
else
|
22
|
+
File.expand_path(results_dirname)
|
23
|
+
end
|
17
24
|
@ob = Orgmode::Parser.new(File.read(dotorg)).babelize
|
18
25
|
@babel = nil
|
19
26
|
@logger = Logger.new(options['--log'] || STDOUT)
|
@@ -40,14 +47,15 @@ module OrgConverge
|
|
40
47
|
|
41
48
|
def converge!
|
42
49
|
tangle!
|
50
|
+
runmode = @options['--runmode'] || ob.in_buffer_settings['RUNMODE']
|
43
51
|
case
|
44
|
-
when @options['--runmode']
|
45
|
-
dispatch_runmode(@options['--runmode'])
|
46
52
|
when @options['--name']
|
47
|
-
|
53
|
+
if runmode == 'sequentially'
|
54
|
+
run_matching_blocks_sequentially!
|
55
|
+
else
|
56
|
+
run_matching_blocks!
|
57
|
+
end
|
48
58
|
else
|
49
|
-
# Try to find one in the buffer
|
50
|
-
runmode = ob.in_buffer_settings['RUNMODE']
|
51
59
|
dispatch_runmode(runmode)
|
52
60
|
end
|
53
61
|
end
|
@@ -60,6 +68,8 @@ module OrgConverge
|
|
60
68
|
run_blocks_sequentially!
|
61
69
|
when 'chained', 'chain', 'tasks'
|
62
70
|
run_blocks_chain!
|
71
|
+
when 'spec'
|
72
|
+
run_against_blocks_results!
|
63
73
|
else # parallel by default
|
64
74
|
run_blocks_in_parallel!
|
65
75
|
end
|
@@ -177,8 +187,135 @@ module OrgConverge
|
|
177
187
|
logger.info "Run has completed successfully.".fg 'green'
|
178
188
|
end
|
179
189
|
|
180
|
-
def
|
181
|
-
|
190
|
+
def run_matching_blocks_sequentially!
|
191
|
+
babel.tangle_runnable_blocks!(@run_dir)
|
192
|
+
|
193
|
+
runlist_stack = []
|
194
|
+
scripts = babel.ob.scripts.select {|k, h| h[:header][:name] =~ Regexp.new(@options['--name']) }
|
195
|
+
scripts.each do |key, script|
|
196
|
+
runlist_stack << [key, script]
|
197
|
+
end
|
198
|
+
|
199
|
+
while not runlist_stack.empty?
|
200
|
+
key, script = runlist_stack.shift
|
201
|
+
|
202
|
+
# Decision: Only run blocks which have a name
|
203
|
+
next unless script[:header][:name]
|
204
|
+
|
205
|
+
display_name = script[:header][:name]
|
206
|
+
with_running_engine do |engine|
|
207
|
+
file = File.expand_path("#{@run_dir}/#{key}")
|
208
|
+
cmd = "#{script[:lang]} #{file}"
|
209
|
+
engine.register display_name, cmd, { :cwd => @root_dir, :logger => logger }
|
210
|
+
end
|
211
|
+
end
|
212
|
+
logger.info "Run has completed successfully.".fg 'green'
|
213
|
+
end
|
214
|
+
|
215
|
+
def run_against_blocks_results!
|
216
|
+
require 'diff/lcs'
|
217
|
+
require 'diff/lcs/hunk'
|
218
|
+
|
219
|
+
succeeded = []
|
220
|
+
failed = []
|
221
|
+
|
222
|
+
logger.info "Runmode: spec"
|
223
|
+
runlist_stack = []
|
224
|
+
scripts = if @options['--name']
|
225
|
+
babel.ob.scripts.select {|k, h| h[:header][:name] =~ Regexp.new(@options['--name']) }
|
226
|
+
else
|
227
|
+
babel.ob.scripts
|
228
|
+
end
|
229
|
+
scripts.each { |key, script| runlist_stack << [key, script] }
|
230
|
+
|
231
|
+
babel.tangle_runnable_blocks!(@run_dir)
|
232
|
+
FileUtils.mkdir_p(@results_dir)
|
233
|
+
|
234
|
+
while not runlist_stack.empty?
|
235
|
+
key, script = runlist_stack.shift
|
236
|
+
|
237
|
+
# Decision: Only run blocks which have a name
|
238
|
+
next unless script[:header][:name]
|
239
|
+
|
240
|
+
display_name = script[:header][:name]
|
241
|
+
script_file = File.expand_path("#{@run_dir}/#{key}")
|
242
|
+
results_file = File.expand_path("#{@results_dir}/#{key}")
|
243
|
+
cmd = "#{script[:lang]} #{script_file}"
|
244
|
+
|
245
|
+
with_running_engine(:runmode => 'spec', :results_dir => @results_dir) \
|
246
|
+
do |engine|
|
247
|
+
engine.register display_name, cmd, {
|
248
|
+
:cwd => @root_dir,
|
249
|
+
:logger => logger,
|
250
|
+
:results => results_file
|
251
|
+
}
|
252
|
+
end
|
253
|
+
|
254
|
+
if scripts[:results]
|
255
|
+
print "Checking results from '#{display_name.fg 'yellow'}' code block:\t"
|
256
|
+
expected_lines = script[:results].split("\n").map! {|e| e.chomp }
|
257
|
+
actual_lines = File.open(results_file).read.split("\n").map! {|e| e.chomp }
|
258
|
+
|
259
|
+
output_diff = diff(expected_lines, actual_lines)
|
260
|
+
if output_diff.empty?
|
261
|
+
puts "OK".fg 'green'
|
262
|
+
succeeded << display_name
|
263
|
+
else
|
264
|
+
puts "DIFF".fg 'red'
|
265
|
+
puts output_diff.fg 'red'
|
266
|
+
failed << display_name
|
267
|
+
end
|
268
|
+
end
|
269
|
+
end
|
270
|
+
|
271
|
+
if failed.count > 0
|
272
|
+
puts ''
|
273
|
+
puts 'Failed code blocks:'.fg 'red'
|
274
|
+
failed.each do |name|
|
275
|
+
puts " - #{name.fg 'yellow'}"
|
276
|
+
end
|
277
|
+
puts ''
|
278
|
+
end
|
279
|
+
|
280
|
+
puts "#{succeeded.count + failed.count} examples, #{failed.count} failures".fg 'green'
|
281
|
+
exit 1 if failed.count > 0
|
282
|
+
end
|
283
|
+
|
284
|
+
def diff(expected_lines, actual_lines)
|
285
|
+
output = ""
|
286
|
+
file_length_difference = 0
|
287
|
+
|
288
|
+
diffs = Diff::LCS.diff(expected_lines, actual_lines)
|
289
|
+
hunks = diffs.map do |piece|
|
290
|
+
Diff::LCS::Hunk.new(
|
291
|
+
expected_lines, actual_lines, piece, 3, 0
|
292
|
+
).tap do |h|
|
293
|
+
file_length_difference = h.file_length_difference
|
294
|
+
end
|
295
|
+
end
|
296
|
+
|
297
|
+
hunks.each_cons(2) do |prev_hunk, current_hunk|
|
298
|
+
begin
|
299
|
+
if current_hunk.overlaps?(prev_hunk)
|
300
|
+
current_hunk.merge(prev_hunk)
|
301
|
+
else
|
302
|
+
output << prev_hunk.diff(:unified).to_s
|
303
|
+
end
|
304
|
+
rescue => e
|
305
|
+
end
|
306
|
+
end
|
307
|
+
|
308
|
+
if hunks.last
|
309
|
+
output << hunks.last.diff(:unified).to_s
|
310
|
+
end
|
311
|
+
|
312
|
+
output
|
313
|
+
end
|
314
|
+
|
315
|
+
def with_running_engine(opts={})
|
316
|
+
default_options = { :logger => @logger, :babel => @babel }
|
317
|
+
options = default_options.merge!(opts)
|
318
|
+
engine = OrgConverge::Engine.new(options)
|
182
319
|
yield engine
|
183
320
|
engine.start
|
184
321
|
end
|
data/lib/org-converge/engine.rb
CHANGED
@@ -16,8 +16,9 @@ module OrgConverge
|
|
16
16
|
|
17
17
|
def initialize(options={})
|
18
18
|
super(options)
|
19
|
-
@logger
|
20
|
-
@babel
|
19
|
+
@logger = options[:logger] || Logger.new(STDOUT)
|
20
|
+
@babel = options[:babel]
|
21
|
+
@runmode = options[:runmode]
|
21
22
|
end
|
22
23
|
|
23
24
|
# We allow other processes to exit with 0 status
|
@@ -37,11 +38,16 @@ module OrgConverge
|
|
37
38
|
@processes.each do |process|
|
38
39
|
reader, writer = create_pipe
|
39
40
|
begin
|
41
|
+
# In case of spec mode, we need to redirect the output to a results file instead
|
42
|
+
writer = File.open(process.options[:results], 'a') if @runmode == 'spec'
|
40
43
|
pid = process.run(:output => writer)
|
41
44
|
@names[process] = "#{@names[process]}.#{pid}"
|
42
|
-
|
45
|
+
|
46
|
+
# NOTE: In spec mode we need to be more strict on what is flushed by the engine
|
47
|
+
# because we will be comparing the output
|
48
|
+
writer.puts "started with pid #{pid}" unless @runmode == 'spec'
|
43
49
|
rescue Errno::ENOENT
|
44
|
-
writer.puts "unknown command: #{process.command}"
|
50
|
+
writer.puts "unknown command: #{process.command}" unless @runmode == 'spec'
|
45
51
|
end
|
46
52
|
@running[pid] = [process]
|
47
53
|
@readers[pid] = reader
|
@@ -63,8 +69,9 @@ module OrgConverge
|
|
63
69
|
output = "#{pad_process_name(ps)}(#{pid})".fg get_color_for_pid(pid.to_i)
|
64
70
|
output += " -- "
|
65
71
|
output += message
|
66
|
-
|
67
|
-
# the name of the process does not appear
|
72
|
+
|
73
|
+
# FIXME: When the process has stopped already, the name of the process does not appear
|
74
|
+
# (which means that this approach is wrong from the beginning probably)
|
68
75
|
logger.info output
|
69
76
|
end
|
70
77
|
rescue Errno::EPIPE
|
@@ -86,7 +93,21 @@ module OrgConverge
|
|
86
93
|
def get_color_for_pid(pid)
|
87
94
|
RAINBOW[pid % 7]
|
88
95
|
end
|
96
|
+
|
97
|
+
def watch_for_termination
|
98
|
+
pid, status = Process.wait2
|
99
|
+
output_with_mutex name_for(pid), termination_message_for(status) unless @runmode == 'spec'
|
100
|
+
@running.delete(pid)
|
101
|
+
yield if block_given?
|
102
|
+
pid
|
103
|
+
rescue Errno::ECHILD
|
104
|
+
end
|
89
105
|
end
|
90
106
|
|
91
|
-
|
107
|
+
# Need to expose the options to make the process be aware
|
108
|
+
# of the possible running mode (specially spec mode)
|
109
|
+
# and where to put the results output
|
110
|
+
class CodeBlockProcess < Foreman::Process
|
111
|
+
attr_reader :options
|
112
|
+
end
|
92
113
|
end
|
data/lib/org-converge/version.rb
CHANGED
data/org-converge.gemspec
CHANGED
@@ -15,7 +15,7 @@ Gem::Specification.new do |gem|
|
|
15
15
|
gem.require_paths = ["lib"]
|
16
16
|
gem.version = OrgConverge::VERSION
|
17
17
|
gem.add_runtime_dependency('docopt', '~> 0.5.0')
|
18
|
-
gem.add_runtime_dependency('org-ruby', '~> 0.9.
|
18
|
+
gem.add_runtime_dependency('org-ruby', '~> 0.9.6')
|
19
19
|
gem.add_runtime_dependency('foreman', '~> 0.63.0')
|
20
20
|
gem.add_runtime_dependency('tco', '~> 0.1.0')
|
21
21
|
gem.add_runtime_dependency('rake', '~> 10.3')
|
@@ -0,0 +1,62 @@
|
|
1
|
+
# -*- mode: org; mode: auto-fill; -*-
|
2
|
+
#+TITLE: Expected results example
|
3
|
+
#+runmode: spec
|
4
|
+
|
5
|
+
Given a literate description of the run along with the expected
|
6
|
+
results, we can try checking whether the contents from the code block
|
7
|
+
are evaling into what is being documented in the results block.
|
8
|
+
|
9
|
+
#+name: hello
|
10
|
+
#+begin_src sh :results output code :exports both
|
11
|
+
for i in `seq 20`; do
|
12
|
+
echo "hello"
|
13
|
+
done
|
14
|
+
#+end_src
|
15
|
+
|
16
|
+
#+RESULTS: hello
|
17
|
+
#+BEGIN_SRC sh
|
18
|
+
hello
|
19
|
+
hello
|
20
|
+
hello
|
21
|
+
hello
|
22
|
+
hello
|
23
|
+
hello
|
24
|
+
hello
|
25
|
+
hello
|
26
|
+
hello
|
27
|
+
hello
|
28
|
+
hello
|
29
|
+
hello
|
30
|
+
hello
|
31
|
+
hello
|
32
|
+
hello
|
33
|
+
hello
|
34
|
+
hello
|
35
|
+
hello
|
36
|
+
hello
|
37
|
+
hello
|
38
|
+
#+END_SRC
|
39
|
+
|
40
|
+
Something else.
|
41
|
+
|
42
|
+
#+name: hola
|
43
|
+
#+begin_src ruby :results output code :exports both
|
44
|
+
10.times do
|
45
|
+
puts "hola"
|
46
|
+
end
|
47
|
+
#+end_src
|
48
|
+
|
49
|
+
#+RESULTS: hola
|
50
|
+
#+BEGIN_SRC ruby
|
51
|
+
hola
|
52
|
+
hola
|
53
|
+
hola
|
54
|
+
hola
|
55
|
+
hola
|
56
|
+
hola
|
57
|
+
hola
|
58
|
+
hola
|
59
|
+
hola
|
60
|
+
hola
|
61
|
+
#+END_SRC
|
62
|
+
|
@@ -0,0 +1,62 @@
|
|
1
|
+
# -*- mode: org; mode: auto-fill; -*-
|
2
|
+
#+TITLE: Expected results example
|
3
|
+
|
4
|
+
#+name: hello_test
|
5
|
+
#+begin_src ruby :results output
|
6
|
+
10.times do
|
7
|
+
puts "hola"
|
8
|
+
end
|
9
|
+
#+end_src
|
10
|
+
|
11
|
+
#+RESULTS: hello_test
|
12
|
+
#+begin_example
|
13
|
+
hola
|
14
|
+
hola
|
15
|
+
hola
|
16
|
+
hola
|
17
|
+
hola
|
18
|
+
hola
|
19
|
+
hola
|
20
|
+
hola
|
21
|
+
hola
|
22
|
+
hola
|
23
|
+
#+end_example
|
24
|
+
|
25
|
+
This one would fail near christmas, outside of Japan,
|
26
|
+
when the site is down, or there is a change in the rss.xml being provided.
|
27
|
+
|
28
|
+
#+name: christmas_check
|
29
|
+
#+begin_src sh :results output
|
30
|
+
curl https://isitchristmas.com/rss.xml 2> /dev/null | grep title
|
31
|
+
# curl isitchristmastyet.com 2> /dev/null
|
32
|
+
#+end_src
|
33
|
+
|
34
|
+
#+RESULTS: christmas_check
|
35
|
+
#+begin_example
|
36
|
+
<title>Is It Christmas?</title>
|
37
|
+
<title>いいえ</title>
|
38
|
+
<title>いいえ</title>
|
39
|
+
<title>いいえ</title>
|
40
|
+
<title>いいえ</title>
|
41
|
+
<title>いいえ</title>
|
42
|
+
<title>いいえ</title>
|
43
|
+
<title>いいえ</title>
|
44
|
+
<title>いいえ</title>
|
45
|
+
<title>いいえ</title>
|
46
|
+
<title>いいえ</title>
|
47
|
+
#+end_example
|
48
|
+
|
49
|
+
#+RESULTS: fetch_google
|
50
|
+
#+begin_example
|
51
|
+
<title>Is It Christmas?</title>
|
52
|
+
<title>いいえ</title>
|
53
|
+
<title>いいえ</title>
|
54
|
+
<title>いいえ</title>
|
55
|
+
<title>いいえ</title>
|
56
|
+
<title>いいえ</title>
|
57
|
+
<title>いいえ</title>
|
58
|
+
<title>いいえ</title>
|
59
|
+
<title>いいえ</title>
|
60
|
+
<title>いいえ</title>
|
61
|
+
<title>いいえ</title>
|
62
|
+
#+end_example
|
data/spec/converge_spec.rb
CHANGED
@@ -105,4 +105,30 @@ describe OrgConverge::Command do
|
|
105
105
|
expected_contents = "init\n0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11"
|
106
106
|
File.read(File.join(example_dir, 'out.log')).should == expected_contents
|
107
107
|
end
|
108
|
+
|
109
|
+
it "should run 'expected_results' with src blocks" do
|
110
|
+
example_dir = File.join(EXAMPLES_DIR, 'expected_results')
|
111
|
+
setup_file = File.join(example_dir, 'spec.org')
|
112
|
+
|
113
|
+
o = OrgConverge::Command.new({
|
114
|
+
'<org_file>' => setup_file,
|
115
|
+
'--root-dir' => example_dir,
|
116
|
+
'--runmode' => 'spec'
|
117
|
+
})
|
118
|
+
success = o.execute!
|
119
|
+
success.should == true
|
120
|
+
end
|
121
|
+
|
122
|
+
it "should run 'expected_results' with example blocks" do
|
123
|
+
example_dir = File.join(EXAMPLES_DIR, 'expected_results')
|
124
|
+
setup_file = File.join(example_dir, 'spec2.org')
|
125
|
+
|
126
|
+
o = OrgConverge::Command.new({
|
127
|
+
'<org_file>' => setup_file,
|
128
|
+
'--root-dir' => example_dir,
|
129
|
+
'--runmode' => 'spec'
|
130
|
+
})
|
131
|
+
success = o.execute!
|
132
|
+
success.should == true
|
133
|
+
end
|
108
134
|
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: org-converge
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.8
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -9,11 +9,11 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2014-05-
|
12
|
+
date: 2014-05-05 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: docopt
|
16
|
-
requirement: &
|
16
|
+
requirement: &70238558687960 !ruby/object:Gem::Requirement
|
17
17
|
none: false
|
18
18
|
requirements:
|
19
19
|
- - ~>
|
@@ -21,21 +21,21 @@ dependencies:
|
|
21
21
|
version: 0.5.0
|
22
22
|
type: :runtime
|
23
23
|
prerelease: false
|
24
|
-
version_requirements: *
|
24
|
+
version_requirements: *70238558687960
|
25
25
|
- !ruby/object:Gem::Dependency
|
26
26
|
name: org-ruby
|
27
|
-
requirement: &
|
27
|
+
requirement: &70238558687480 !ruby/object:Gem::Requirement
|
28
28
|
none: false
|
29
29
|
requirements:
|
30
30
|
- - ~>
|
31
31
|
- !ruby/object:Gem::Version
|
32
|
-
version: 0.9.
|
32
|
+
version: 0.9.6
|
33
33
|
type: :runtime
|
34
34
|
prerelease: false
|
35
|
-
version_requirements: *
|
35
|
+
version_requirements: *70238558687480
|
36
36
|
- !ruby/object:Gem::Dependency
|
37
37
|
name: foreman
|
38
|
-
requirement: &
|
38
|
+
requirement: &70238558687020 !ruby/object:Gem::Requirement
|
39
39
|
none: false
|
40
40
|
requirements:
|
41
41
|
- - ~>
|
@@ -43,10 +43,10 @@ dependencies:
|
|
43
43
|
version: 0.63.0
|
44
44
|
type: :runtime
|
45
45
|
prerelease: false
|
46
|
-
version_requirements: *
|
46
|
+
version_requirements: *70238558687020
|
47
47
|
- !ruby/object:Gem::Dependency
|
48
48
|
name: tco
|
49
|
-
requirement: &
|
49
|
+
requirement: &70238558686560 !ruby/object:Gem::Requirement
|
50
50
|
none: false
|
51
51
|
requirements:
|
52
52
|
- - ~>
|
@@ -54,10 +54,10 @@ dependencies:
|
|
54
54
|
version: 0.1.0
|
55
55
|
type: :runtime
|
56
56
|
prerelease: false
|
57
|
-
version_requirements: *
|
57
|
+
version_requirements: *70238558686560
|
58
58
|
- !ruby/object:Gem::Dependency
|
59
59
|
name: rake
|
60
|
-
requirement: &
|
60
|
+
requirement: &70238558686100 !ruby/object:Gem::Requirement
|
61
61
|
none: false
|
62
62
|
requirements:
|
63
63
|
- - ~>
|
@@ -65,13 +65,14 @@ dependencies:
|
|
65
65
|
version: '10.3'
|
66
66
|
type: :runtime
|
67
67
|
prerelease: false
|
68
|
-
version_requirements: *
|
68
|
+
version_requirements: *70238558686100
|
69
69
|
description: A light configuration management tool for Org mode
|
70
70
|
email:
|
71
71
|
- waldemar.quevedo@gmail.com
|
72
72
|
executables:
|
73
73
|
- org-converge
|
74
74
|
- org-run
|
75
|
+
- org-spec
|
75
76
|
- org-tangle
|
76
77
|
extensions: []
|
77
78
|
extra_rdoc_files: []
|
@@ -85,6 +86,7 @@ files:
|
|
85
86
|
- TODO
|
86
87
|
- bin/org-converge
|
87
88
|
- bin/org-run
|
89
|
+
- bin/org-spec
|
88
90
|
- bin/org-tangle
|
89
91
|
- examples/apt-get-install/setup.org
|
90
92
|
- examples/fluentd/setup.org
|
@@ -101,6 +103,8 @@ files:
|
|
101
103
|
- spec/converge_examples/basic_tangle/conf.yml.expected
|
102
104
|
- spec/converge_examples/basic_tangle/setup.org
|
103
105
|
- spec/converge_examples/commented_block/run.org
|
106
|
+
- spec/converge_examples/expected_results/spec.org
|
107
|
+
- spec/converge_examples/expected_results/spec2.org
|
104
108
|
- spec/converge_examples/linked_tasks/tasks.org
|
105
109
|
- spec/converge_examples/runlist_example/setup.org
|
106
110
|
- spec/converge_examples/specified_block/run.org
|
@@ -136,6 +140,8 @@ test_files:
|
|
136
140
|
- spec/converge_examples/basic_tangle/conf.yml.expected
|
137
141
|
- spec/converge_examples/basic_tangle/setup.org
|
138
142
|
- spec/converge_examples/commented_block/run.org
|
143
|
+
- spec/converge_examples/expected_results/spec.org
|
144
|
+
- spec/converge_examples/expected_results/spec2.org
|
139
145
|
- spec/converge_examples/linked_tasks/tasks.org
|
140
146
|
- spec/converge_examples/runlist_example/setup.org
|
141
147
|
- spec/converge_examples/specified_block/run.org
|